-- Logs begin at Wed 2022-08-24 13:36:22 UTC, end at Wed 2022-08-24 15:02:21 UTC. -- Aug 24 14:10:25.392628 dv-cinder-756709-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 14:10:25 2022] *** Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: nodename: dv-cinder-756709-6-iscsi Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: machine: x86_64 Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: clock source: unix Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: pcre jit disabled Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: detected number of CPU cores: 4 Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: current working directory: / Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: detected binary path: /usr/bin/uwsgi-core Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: your processes number limit is 47680 Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: your memory page size is 4096 bytes Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: detected max file descriptor number: 2048 Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: lock engine: pthread robust mutexes Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: thunder lock: enabled Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 14:10:25.410854 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 14:10:25.447210 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: Python main interpreter initialized at 0x55d2d46199c0 Aug 24 14:10:25.447210 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: python threads support enabled Aug 24 14:10:25.447210 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: your server socket listen backlog is limited to 100 connections Aug 24 14:10:25.447210 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: your mercy for graceful operations on workers is 90 seconds Aug 24 14:10:25.447727 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 14:10:25.447727 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: *** Operational MODE: preforking *** Aug 24 14:10:25.447973 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: *** uWSGI is running in multiple interpreter mode *** Aug 24 14:10:25.447973 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: spawned uWSGI master process (pid: 81710) Aug 24 14:10:25.448253 dv-cinder-756709-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 14:10:25.449351 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: spawned uWSGI worker 1 (pid: 81724, cores: 1) Aug 24 14:10:25.449351 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: spawned uWSGI worker 2 (pid: 81725, cores: 1) Aug 24 14:10:25.449351 dv-cinder-756709-6-iscsi devstack@keystone.service[81710]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 14:10:29.982719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 14:10:29.982719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: from cryptography.utils import int_from_bytes Aug 24 14:10:29.983679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 14:10:29.983679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: from cryptography.utils import int_from_bytes Aug 24 14:10:30.013270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 14:10:30.013270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: from cryptography.utils import int_from_bytes Aug 24 14:10:30.013936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 14:10:30.013936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: from cryptography.utils import int_from_bytes Aug 24 14:10:30.044663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] command line args: [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ================================================================================ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] admin_token = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config_dir = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config_source = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.048246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] debug = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.049327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.049327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.049327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.049327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.049327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.049327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.049327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_config_append = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_dir = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.051380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.052971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.053272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] publish_errors = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.053384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.053610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.053826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.054053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.054314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.054532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] standard_threads = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.054738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.054979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.055223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.055398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_journal = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.055604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_json = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.055807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_stderr = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.055949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_syslog = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.056150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.056413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.056650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.056896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.057394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.057635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.057885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.058156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.058407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.058655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.058906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.059141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.059390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.059625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.059876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.060127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.060365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.060621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.060876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.061108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.061346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.061602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.061867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.062218 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.062469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.062741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.062936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.063678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.063916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.064177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.064394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.064626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.064859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.065101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.065347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.065590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.065796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 14:10:30.066012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.066199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 14:10:30.066394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.066579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] command line args: [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 14:10:30.066980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.067160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 14:10:30.067353 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.067527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ================================================================================ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 14:10:30.067717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.067902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] admin_token = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.068096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.068281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config_dir = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.068462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.068650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.068803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.068984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] config_source = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.069289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.069477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] debug = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.069477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.069882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.070087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.070285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.070472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.070644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.070830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.071017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.071204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.071397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.071579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.071762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.071937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_config_append = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.072129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.072309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.072309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_dir = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.072673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.072850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.073025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.073190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.073386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.073603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.073786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.073974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.074200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.074367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.074544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.074725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.074893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.075069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.075243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.075409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.075596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.075760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.075928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.076093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.076273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.076436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.076716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.076896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.077107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.077283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.077461 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.077637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.077827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.078016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.078127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.078226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.081928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.081928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.082262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.082346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.082421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] publish_errors = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.082499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.082580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.082653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.082724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.082724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.082907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.083015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.083129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.083223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.083343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.083442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] standard_threads = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.083544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.083623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.083703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.083857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.083936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.084059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.084173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.084251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_journal = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.084320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.084428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_json = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.084428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_stderr = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.084605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.084735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] use_syslog = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.084839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.084942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 14:10:30.085044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.085918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.086925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087603 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087603 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087772 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.087961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088779 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.external = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.088945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.password = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.token = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.089935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.090998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.091977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.092980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.093941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.094991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.095938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.096958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.097887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.098942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.099947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.100970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.101939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.external = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.102989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.password = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] auth.token = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.103937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.104967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.105967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.106090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.106203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.106331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.106447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.106595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.106748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.106854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.107999 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.108148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.108283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.108399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.108554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.108673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.108806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.108935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109286 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.109952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.110964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.111122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.111250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.111416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.111416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.111655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.111892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.112041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.112172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.112278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.112428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.112593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.112748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.112904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.113924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.114108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.120990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.121945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.122944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.123913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.124941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.125012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.125082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.125152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.125222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.125375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.125451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.126935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127284 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.127980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128197 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.128950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.129932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.130969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.131933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.132994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 14:10:30.133626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.133985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.134201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.134403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.134653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.134885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.135190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.135451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.135701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.135951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.136199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.136447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.136694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.136943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.137176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.137426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.137673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.137919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.138188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.138615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.138847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.139097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.139346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.139592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.139839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.140076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.140331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.140584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.140821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.141053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.142099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.142450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.142450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.142684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.143099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.143099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.143545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.143545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.143774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.144000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.144310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.144521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.144948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.144948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.146234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.146234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.146234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.146234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.146234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.157826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.158610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 14:10:30.159142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 14:10:30.162945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.163180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.163409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.171918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.178022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.178262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.178497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.186923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.201384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.201640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.206101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.206334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.272100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.276630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.324820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.325061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.326899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.327137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.493113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.493344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.493583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.493770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.493944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.494198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.497857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.498112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.498344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.498558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.498778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.499019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.529078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.529376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.529739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.530132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.530490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.530838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.531083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.531206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.531600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.532519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.532872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.533228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.533603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.533955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.552809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.553032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.574859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.575034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.575231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.575485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.575593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.575845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.575937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.576198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.576298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.576533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.576622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.576869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.576963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.577202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81725) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 14:10:30.605586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.605897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 14:10:30.606304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.606551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.606651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 14:10:30.606866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 14:10:30.607294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.607380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.607637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 14:10:30.607730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.608062 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.608478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.608570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.608895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.608981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.609296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.626699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.627154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.627588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.628059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.628189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.628681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.629715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.630466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.641060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.644185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.646836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.647290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.647736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.648182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.648620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.649047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.649304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.649489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.649749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.649938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.650210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.650586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.650669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.651040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.651158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.651446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.651533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.651871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.652308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.652743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.653191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.653622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.693322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.695913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.696367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.698874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.701839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.702320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.705447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.705915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.706395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.712812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.713274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.713711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.717901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.718396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.724117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.724335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.724468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.724468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.724731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.724857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.725025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.725310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.725587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.725863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.729514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.729822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.730137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.730507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.730861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.731231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.731557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.731828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.744835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.745145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.749180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.753057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.753371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.754987 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.755385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.757757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.760384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.760992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.764346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.764608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.764764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.764934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.765072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.765206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.765501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.765792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.770328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.774546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.778827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.779340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.779599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.780095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.780236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.780705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.783240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.783782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.784265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.784632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.792048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.793595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.793877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.796563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.796854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.797159 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.797519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.798533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.800030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.800336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.800605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.800866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.808234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.809951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.810242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.811553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.811858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.812176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.812466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.812804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.813436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.813713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.813797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.813891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.814011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.814011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.814281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.827537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.836015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.836015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.836015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.836015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.836015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.836015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.836557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.836557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.837198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.838710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.840187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.840480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.840785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.846964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.848331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.849826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.850300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.852764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.853059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.853187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.853360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.853643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.854671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.856143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.856428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.856712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.862615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.862916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.863197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.864284 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.865769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.868825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.869145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.869445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.869770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.870269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.870571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.870858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.871142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.871427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.871705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.871997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.872270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.872627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.872834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.873106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.873382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.873673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.873943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.878132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.878436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.878719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.886051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81725) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 14:10:30.886348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.886637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.886912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.887181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.887457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.887733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.888010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.888289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.888557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.888835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.889093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.889367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.889641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81725) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 14:10:30.911091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d2d46199c0 pid: 81724 (default app) Aug 24 14:10:30.915115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495a8c1-95de-48ce-bed9-afa300811171 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:30.915380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495a8c1-95de-48ce-bed9-afa300811171 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:30.915665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495a8c1-95de-48ce-bed9-afa300811171 None None] PATH_INFO: `/v3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:30.920004 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Wed Aug 24 14:10:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:30.929155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d2d46199c0 pid: 81725 (default app) Aug 24 14:10:39.193085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0468231-5e73-49ef-bd61-8aa36a213cf1 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:39.205362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0468231-5e73-49ef-bd61-8aa36a213cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:39.205362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0468231-5e73-49ef-bd61-8aa36a213cf1 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:39.205362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 2/2] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:10:39.205676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:39.205676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:39.205676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:39.277624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: INFO dbcounter [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] Registered counter for database keystone Aug 24 14:10:39.303409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG oslo_db.sqlalchemy.engines [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81725) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 14:10:39.310576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writer thread running {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 14:10:39.502848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG passlib.handlers.bcrypt [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81725) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 14:10:39.503189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG passlib.handlers.bcrypt [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81725) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 14:10:39.561821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:39.564609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:39.705719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1/3] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 14:10:40.169322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0468231-5e73-49ef-bd61-8aa36a213cf1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:40.241042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: INFO dbcounter [None req-a0468231-5e73-49ef-bd61-8aa36a213cf1 None None] Registered counter for database keystone Aug 24 14:10:40.262316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG oslo_db.sqlalchemy.engines [None req-a0468231-5e73-49ef-bd61-8aa36a213cf1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81724) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 14:10:40.270191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writer thread running {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 14:10:40.398265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0468231-5e73-49ef-bd61-8aa36a213cf1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:40.402912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bcf36-45c7-42f4-92f5-41561a79d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:40.403672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bcf36-45c7-42f4-92f5-41561a79d914 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:40.403953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bcf36-45c7-42f4-92f5-41561a79d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:40.404221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bcf36-45c7-42f4-92f5-41561a79d914 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:40.414535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bcf36-45c7-42f4-92f5-41561a79d914 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:40.468195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING keystone.common.rbac_enforcer.enforcer [None req-292bcf36-45c7-42f4-92f5-41561a79d914 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 14:10:40.468449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bcf36-45c7-42f4-92f5-41561a79d914 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:40.480405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 3/4] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Aug 24 14:10:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:10:40.487395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:40.503489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ced8de-eeea-4efb-bce5-4241d2bee8fb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:40.508195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836a805-8833-457e-8179-c10943dfa194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:40.509361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836a805-8833-457e-8179-c10943dfa194 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:40.509640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836a805-8833-457e-8179-c10943dfa194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:40.509911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836a805-8833-457e-8179-c10943dfa194 admin admin] PATH_INFO: `/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:40.511691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836a805-8833-457e-8179-c10943dfa194 admin admin] RBAC: Authorizing `identity:get_project(project_id=f90d0e4c756a418fb562e2e8abdd6ef4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:40.566427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8836a805-8833-457e-8179-c10943dfa194 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 14:10:40.566768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836a805-8833-457e-8179-c10943dfa194 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:40.571504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 2/5] 10.222.0.49 () {56 vars in 1207 bytes} [Wed Aug 24 14:10:40 2022] GET /identity/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:42.586682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ccffe-0a01-43b7-89b7-b9620a566f8d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:42.587327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ccffe-0a01-43b7-89b7-b9620a566f8d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:42.587327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ccffe-0a01-43b7-89b7-b9620a566f8d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:42.591737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 4/6] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:42.598575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310c28-0cb5-4345-9226-e3f048a566d0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:42.599117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310c28-0cb5-4345-9226-e3f048a566d0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:42.599117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310c28-0cb5-4345-9226-e3f048a566d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:42.620580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-54310c28-0cb5-4345-9226-e3f048a566d0 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:42.622121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-54310c28-0cb5-4345-9226-e3f048a566d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:42.631149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 3/7] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 14:10:42.904690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ccffe-0a01-43b7-89b7-b9620a566f8d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:42.918889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ccffe-0a01-43b7-89b7-b9620a566f8d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:42.924310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87672cc4-5d2b-464c-a053-c10368da842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:42.924942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672cc4-5d2b-464c-a053-c10368da842f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:42.925223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672cc4-5d2b-464c-a053-c10368da842f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:42.925498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672cc4-5d2b-464c-a053-c10368da842f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:42.934977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87672cc4-5d2b-464c-a053-c10368da842f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:42.936830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87672cc4-5d2b-464c-a053-c10368da842f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:42.947395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 5/8] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Aug 24 14:10:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:10:42.954294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54310c28-0cb5-4345-9226-e3f048a566d0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:42.964714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54310c28-0cb5-4345-9226-e3f048a566d0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:42.970712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d29d1-032a-40e9-8ab3-a60400390898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:42.971417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d29d1-032a-40e9-8ab3-a60400390898 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:42.971700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d29d1-032a-40e9-8ab3-a60400390898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:42.971972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d29d1-032a-40e9-8ab3-a60400390898 admin admin] PATH_INFO: `/v3/users/34ed7567d900480cb00efd1e75c2ddc5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:42.973448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d29d1-032a-40e9-8ab3-a60400390898 admin admin] RBAC: Authorizing `identity:get_user(user_id=34ed7567d900480cb00efd1e75c2ddc5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:42.975390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d29d1-032a-40e9-8ab3-a60400390898 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:42.977187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 4/9] 10.222.0.49 () {56 vars in 1198 bytes} [Wed Aug 24 14:10:42 2022] GET /identity/v3/users/34ed7567d900480cb00efd1e75c2ddc5 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:45.124040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180db6f-161b-4108-adb8-042f4d22ad9c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:45.124633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180db6f-161b-4108-adb8-042f4d22ad9c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:45.124633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180db6f-161b-4108-adb8-042f4d22ad9c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:45.126104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 6/10] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:45.131784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1fd1e-463a-4497-9528-3ab07660e8af None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:45.132600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1fd1e-463a-4497-9528-3ab07660e8af None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:45.132781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1fd1e-463a-4497-9528-3ab07660e8af None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:45.133018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:45.133307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:45.133586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:45.134546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 5/11] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:45.143094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb76b8b-a1be-4dfe-a958-e5b8f3bf4fda None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:45.143513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb76b8b-a1be-4dfe-a958-e5b8f3bf4fda None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:45.143877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb76b8b-a1be-4dfe-a958-e5b8f3bf4fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:45.165657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5fb76b8b-a1be-4dfe-a958-e5b8f3bf4fda None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:45.167117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5fb76b8b-a1be-4dfe-a958-e5b8f3bf4fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:45.185842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG passlib.handlers.bcrypt [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81724) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 14:10:45.199117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG passlib.handlers.bcrypt [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81724) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 14:10:45.240980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:45.242859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:45.279010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:10:45.374306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:45.555041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb76b8b-a1be-4dfe-a958-e5b8f3bf4fda None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:45.582660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:45.678115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb76b8b-a1be-4dfe-a958-e5b8f3bf4fda None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:45.717301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:45.717956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:45.718273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:45.718549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:45.732717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:10:45.742430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:10:45.744084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:10:45.745632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:10:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:10:45.752483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:45.877042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e144caf-0442-4bc7-8eea-80f3da10306d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:45.880160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:45.883884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:45.927310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:45.928492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:45.928855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:45.929573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:45.930483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:10:45.931230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:45.934293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:45.951824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:10:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:45.960506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:45.993480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:46.015159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbbef-0b24-46dd-8d8e-aecff3529507 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:46.067945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:46.069005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:46.069502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:46.070028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:46.144606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30781c11-760f-4999-98ee-3f82c7b1138e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:46.151402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:46.152963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:46.195940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:46.195940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:46.195940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:46.195940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] PATH_INFO: `/v3/users/34ed7567d900480cb00efd1e75c2ddc5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:46.196615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=34ed7567d900480cb00efd1e75c2ddc5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:46.198238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:46.200480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 9/16] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:10:45 2022] GET /identity/v3/users/34ed7567d900480cb00efd1e75c2ddc5 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:46.208410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:46.257950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:46.278788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:46.299978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 8/17] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 14:10:46.306704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:46.380960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b8d4-4c51-4b30-a5de-9a15f3612bc3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:46.430184 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:46.431119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:46.432400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:46.432806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:46.434404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:46.437262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:46.440938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 10/18] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:46.448731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:46.454156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af738f1-c48f-44e5-a0da-90a902b12297 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:46.508489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:46.509327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:46.509730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:46.511199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:46.525849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:46.527590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:10:46.545806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:10:46.547822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:10:46.549518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:10:46.549518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:10:46.549518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:10:46.549518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 9/19] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:10:46.553373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:46.592658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c45e7f-90ef-415c-b674-acb7eb8cb1d9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:46.594662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:46.594838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:46.633239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbe490-2502-4ea5-9973-d11febff11ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:46.633871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe490-2502-4ea5-9973-d11febff11ff None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:46.634178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe490-2502-4ea5-9973-d11febff11ff None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:46.634458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe490-2502-4ea5-9973-d11febff11ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:46.635210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbe490-2502-4ea5-9973-d11febff11ff None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.domain.id=default, user.id=34ed7567d900480cb00efd1e75c2ddc5) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:10:46.635516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbe490-2502-4ea5-9973-d11febff11ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:46.637019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbe490-2502-4ea5-9973-d11febff11ff None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:46.647096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 11/20] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Aug 24 14:10:46 2022] GET /identity/v3/role_assignments?user.id=34ed7567d900480cb00efd1e75c2ddc5&scope.domain.id=default&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 233 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:10:46.725086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b4d36-a768-4bed-8acc-096858ab175b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:46.795224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d1d8e-c310-4f6f-aac6-b765af34d5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:46.796298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d1d8e-c310-4f6f-aac6-b765af34d5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:46.797399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d1d8e-c310-4f6f-aac6-b765af34d5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:46.797899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d1d8e-c310-4f6f-aac6-b765af34d5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:46.799145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d1d8e-c310-4f6f-aac6-b765af34d5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:10:46.799650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d1d8e-c310-4f6f-aac6-b765af34d5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:46.802347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d1d8e-c310-4f6f-aac6-b765af34d5f2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:46.837166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 10/21] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:10:49.491408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1122-4b0d-4192-8a1e-a01ef2a0bf9a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:49.491952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1122-4b0d-4192-8a1e-a01ef2a0bf9a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:49.492379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1122-4b0d-4192-8a1e-a01ef2a0bf9a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:49.494958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 12/22] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:49.502870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f9a97-6cb7-4855-91cf-3281a7bb7691 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:49.503388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f9a97-6cb7-4855-91cf-3281a7bb7691 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:49.503840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f9a97-6cb7-4855-91cf-3281a7bb7691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:49.543617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2f7f9a97-6cb7-4855-91cf-3281a7bb7691 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:49.544857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2f7f9a97-6cb7-4855-91cf-3281a7bb7691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:49.646548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 11/23] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:49.959440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32f16-ef58-4c14-889f-0dfe96f1edb4 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:49.959792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32f16-ef58-4c14-889f-0dfe96f1edb4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:49.959903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32f16-ef58-4c14-889f-0dfe96f1edb4 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:49.961286 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:49.976098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c509b3-91bf-4546-aa3f-6741ed74238c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:49.976417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c509b3-91bf-4546-aa3f-6741ed74238c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:49.976726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c509b3-91bf-4546-aa3f-6741ed74238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.007264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c0c509b3-91bf-4546-aa3f-6741ed74238c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:50.008534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c0c509b3-91bf-4546-aa3f-6741ed74238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:50.037086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:50.115740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 12/25] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:50.221173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ddc4-09a3-48c4-8140-556402c589d8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:50.221173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ddc4-09a3-48c4-8140-556402c589d8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:50.221173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ddc4-09a3-48c4-8140-556402c589d8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.221173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 14/26] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:50.228639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c156ed-79ba-4753-b3b2-82a5252f77b8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:50.228955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c156ed-79ba-4753-b3b2-82a5252f77b8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:50.229202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c156ed-79ba-4753-b3b2-82a5252f77b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.259995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-60c156ed-79ba-4753-b3b2-82a5252f77b8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:50.261247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-60c156ed-79ba-4753-b3b2-82a5252f77b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:50.275972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ddc4-09a3-48c4-8140-556402c589d8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:50.389103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 13/27] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:50.391031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa3a67-c29e-4b67-85f2-31990cdbbb22 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:50.391668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa3a67-c29e-4b67-85f2-31990cdbbb22 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:50.392369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa3a67-c29e-4b67-85f2-31990cdbbb22 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.404160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 14/28] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:50.410217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adde7fd-5da3-4027-8977-d4189cd945db None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:50.410736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adde7fd-5da3-4027-8977-d4189cd945db None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:50.411153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adde7fd-5da3-4027-8977-d4189cd945db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.445007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6adde7fd-5da3-4027-8977-d4189cd945db None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:50.446535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6adde7fd-5da3-4027-8977-d4189cd945db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:50.462562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ddc4-09a3-48c4-8140-556402c589d8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:50.534328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:50.537654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:50.538248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:50.538904 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.552816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 15/29] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:10:50.554434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adde7fd-5da3-4027-8977-d4189cd945db None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:50.570123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:10:50.586823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:10:50.587728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:10:50.588504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:10:50.592918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:50.596000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 15/30] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:10:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:10:50.604086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:50.667479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:50.804495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adde7fd-5da3-4027-8977-d4189cd945db None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:50.860688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a79d-7d3e-4a9e-b06e-e5a18a0cda25 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:50.908165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:50.908797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:50.909075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:50.909345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.910048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:50.911473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:50.948387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:50.949029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:50.949311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:50.949587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:50.950327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:10:50.950623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:50.952125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:50.966857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 16/31] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:10:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:50.973096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:51.104405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa87de-b3b7-4a32-be01-22881f2d3d75 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:51.106367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:51.128957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 16/32] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 24 14:10:50 2022] POST /identity/v3/roles => generated 219 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:10:51.130022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:51.148896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:51.149621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.149905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.150218 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] PATH_INFO: `/v3/users/34ed7567d900480cb00efd1e75c2ddc5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.151498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] RBAC: Authorizing `identity:get_user(user_id=34ed7567d900480cb00efd1e75c2ddc5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:51.153821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:51.156222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:10:50 2022] GET /identity/v3/users/34ed7567d900480cb00efd1e75c2ddc5 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:10:51.163010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:51.292281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:51.326779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe91ba-46af-4557-9315-c5f46d6279be None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:51.339712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cce41b-2d3d-400e-8798-23da9884f80b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:51.381963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e0916-0525-4fc4-9310-3e4815c7a6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:51.382737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0916-0525-4fc4-9310-3e4815c7a6a9 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.383025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0916-0525-4fc4-9310-3e4815c7a6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.383296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e0916-0525-4fc4-9310-3e4815c7a6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.384002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e0916-0525-4fc4-9310-3e4815c7a6a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:51.385599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e0916-0525-4fc4-9310-3e4815c7a6a9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:51.397260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:51.397943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.398249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.398524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.422701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:51.424296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:10:51.443467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:10:51.445334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:10:51.447698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:10:51.447698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:10:51.447698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:10:51.447698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 18/34] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:10:51.447698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:51.461367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:51.466038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 17/35] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 24 14:10:51 2022] POST /identity/v3/roles => generated 225 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:10:51.470562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813859bb-3182-4fbe-980d-a35d1799fca8 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.470562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813859bb-3182-4fbe-980d-a35d1799fca8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.470562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813859bb-3182-4fbe-980d-a35d1799fca8 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.477101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 18/36] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:51.477101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483aef2-1545-4145-ad04-28241322025a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.477101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483aef2-1545-4145-ad04-28241322025a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.477101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483aef2-1545-4145-ad04-28241322025a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.478433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 19/37] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:51.484689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483aef2-1545-4145-ad04-28241322025a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:51.630185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d715da-76a6-4747-a7e1-bc41bb71f8fb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:51.693168 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae239ef7-0dfa-43ee-9aaa-2dcdde373e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:51.694246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae239ef7-0dfa-43ee-9aaa-2dcdde373e60 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.694695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae239ef7-0dfa-43ee-9aaa-2dcdde373e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.695181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae239ef7-0dfa-43ee-9aaa-2dcdde373e60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.697202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae239ef7-0dfa-43ee-9aaa-2dcdde373e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:51.699687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae239ef7-0dfa-43ee-9aaa-2dcdde373e60 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:51.704490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 19/38] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:51.706298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cadac3-3b7f-4980-aad7-006f604c6c67 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.706747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cadac3-3b7f-4980-aad7-006f604c6c67 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.707225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cadac3-3b7f-4980-aad7-006f604c6c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.743871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-08cadac3-3b7f-4980-aad7-006f604c6c67 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:51.745953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-08cadac3-3b7f-4980-aad7-006f604c6c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:51.788436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483aef2-1545-4145-ad04-28241322025a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:51.833384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:51.834520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.835008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.835501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.836668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:10:51.837205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:51.839967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:51.842910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:51.852565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 20/39] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:51.868468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9cd07-7404-4cbe-acbe-209880684669 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:51.868468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9cd07-7404-4cbe-acbe-209880684669 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:51.868468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9cd07-7404-4cbe-acbe-209880684669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:51.868936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 20/40] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:51.868936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:51.876655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:51.886493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d2b9cd07-7404-4cbe-acbe-209880684669 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:51.888508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d2b9cd07-7404-4cbe-acbe-209880684669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:52.026776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b43489-d8e2-4588-aea9-07a7c869c199 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:52.029711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 21/41] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:52.034309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9cd07-7404-4cbe-acbe-209880684669 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:52.110104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:52.111383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:52.112182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:52.112761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] PATH_INFO: `/v3/domains/default/users/34ed7567d900480cb00efd1e75c2ddc5/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:52.116770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=34ed7567d900480cb00efd1e75c2ddc5, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:52.120243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:52.205667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9cd07-7404-4cbe-acbe-209880684669 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:52.252468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 21/42] 10.222.0.49 () {58 vars in 1361 bytes} [Wed Aug 24 14:10:51 2022] PUT /identity/v3/domains/default/users/34ed7567d900480cb00efd1e75c2ddc5/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:10:52.276985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:52.277888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:52.278215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:52.278490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:52.279179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:52.281087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:52.290293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:52.319729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:52.395555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:52.486140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76749690-027c-4167-b406-740ef097b6b1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:52.556728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:52.557808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:52.558309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:52.558671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:52.560480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:52.562408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:52.564329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 22/43] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:10:52.565389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:52.607533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 22/44] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:10:52 2022] POST /identity/v3/projects => generated 301 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:10:52.611199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:52.748098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef1e9-22bd-4fbd-a899-4449289e14e8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:52.787177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdce0e-5464-4572-8a47-7598f994e104 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:52.787716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:52.820019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9c978-cc52-41cf-89ea-9d8bb75cbc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:52.820646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9c978-cc52-41cf-89ea-9d8bb75cbc12 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:52.820876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9c978-cc52-41cf-89ea-9d8bb75cbc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:52.821032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9c978-cc52-41cf-89ea-9d8bb75cbc12 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:52.821919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9c978-cc52-41cf-89ea-9d8bb75cbc12 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:52.824114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9c978-cc52-41cf-89ea-9d8bb75cbc12 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:52.843533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653787e7-4084-436e-95ec-6c98d51e0c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:52.847322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653787e7-4084-436e-95ec-6c98d51e0c14 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:52.847322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653787e7-4084-436e-95ec-6c98d51e0c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:52.847322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653787e7-4084-436e-95ec-6c98d51e0c14 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:52.847322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653787e7-4084-436e-95ec-6c98d51e0c14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:52.848013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653787e7-4084-436e-95ec-6c98d51e0c14 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:52.921431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 23/45] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 24 14:10:52 2022] POST /identity/v3/roles => generated 223 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:10:52.952068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 23/46] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:10:52 2022] POST /identity/v3/projects => generated 312 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:10:54.424404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5cb9-2142-4882-8d7a-71a6c705e44a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:54.424944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5cb9-2142-4882-8d7a-71a6c705e44a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:54.424944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5cb9-2142-4882-8d7a-71a6c705e44a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:54.426637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 24/47] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:54.432179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85bfe-00bb-48c9-aa48-2fb80d0e42ff None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:54.432710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85bfe-00bb-48c9-aa48-2fb80d0e42ff None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:54.432797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85bfe-00bb-48c9-aa48-2fb80d0e42ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:54.453397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-19f85bfe-00bb-48c9-aa48-2fb80d0e42ff None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:54.455447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-19f85bfe-00bb-48c9-aa48-2fb80d0e42ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:54.496868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:54.536142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 24/48] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:54.825544 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5cb9-2142-4882-8d7a-71a6c705e44a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:54.960613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5cb9-2142-4882-8d7a-71a6c705e44a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:55.003103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4f640-d988-460f-b69a-181958a0a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:55.003757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4f640-d988-460f-b69a-181958a0a422 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.004042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4f640-d988-460f-b69a-181958a0a422 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.004319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4f640-d988-460f-b69a-181958a0a422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.017481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4f640-d988-460f-b69a-181958a0a422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e8f4f640-d988-460f-b69a-181958a0a422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:10:55.022507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:10:55.023768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:10:55.024794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 25/49] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:10:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:10:55.033409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85bfe-00bb-48c9-aa48-2fb80d0e42ff None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:55.083506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a61d01-8302-4c27-8249-2d4c4fb2b80f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.083836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a61d01-8302-4c27-8249-2d4c4fb2b80f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.084288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a61d01-8302-4c27-8249-2d4c4fb2b80f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.086154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 26/50] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:10:55.092610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8df7-fb7c-4f7d-adf0-e7405b223ac8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.093068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8df7-fb7c-4f7d-adf0-e7405b223ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.093463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8df7-fb7c-4f7d-adf0-e7405b223ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.114308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2c1d8df7-fb7c-4f7d-adf0-e7405b223ac8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:55.115906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2c1d8df7-fb7c-4f7d-adf0-e7405b223ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:55.155762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:55.191052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 27/51] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:10:55.213189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85bfe-00bb-48c9-aa48-2fb80d0e42ff None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:55.261488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:55.262295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.262638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.263003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.263766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:10:55.264128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:55.266362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:55.285260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 25/52] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:55.287432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d8df7-fb7c-4f7d-adf0-e7405b223ac8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:55.448546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d8df7-fb7c-4f7d-adf0-e7405b223ac8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:55.473738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:55.507517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:55.508565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.508943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.509222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] PATH_INFO: `/v3/users/34ed7567d900480cb00efd1e75c2ddc5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.511110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] RBAC: Authorizing `identity:get_user(user_id=34ed7567d900480cb00efd1e75c2ddc5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:55.511323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:55.513245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:55.516058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 28/53] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:10:55 2022] GET /identity/v3/users/34ed7567d900480cb00efd1e75c2ddc5 => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:55.522897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:55.592483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:55.603925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3d78b-b791-417d-96c8-9b8d18b311ee None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:55.647089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:55.647907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.648375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.648831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.650894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:55.653418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:55.657332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 26/54] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:10:55.663905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:55.680998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf2c58-95d0-44c9-8ec0-add5e1db2d8c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:55.723792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:55.724448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.724728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.725004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.726248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:55.727754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:55.729430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 29/55] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:55.736550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:55.804363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:55.807514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df881-a8e3-4c21-960d-6e84cb0fbb09 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:55.849921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd10ec-627c-4ee6-b77c-9d074e3d8988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:55.850664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd10ec-627c-4ee6-b77c-9d074e3d8988 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.850940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd10ec-627c-4ee6-b77c-9d074e3d8988 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.851214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd10ec-627c-4ee6-b77c-9d074e3d8988 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.851877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd10ec-627c-4ee6-b77c-9d074e3d8988 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:55.853340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd10ec-627c-4ee6-b77c-9d074e3d8988 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:55.854357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f253de-d6f5-4d0e-886c-7dfc45169569 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:55.866374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:55.893687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a2aee-eb58-41af-b615-44b2511e32ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:55.894394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a2aee-eb58-41af-b615-44b2511e32ee None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:55.894673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a2aee-eb58-41af-b615-44b2511e32ee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:55.894943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a2aee-eb58-41af-b615-44b2511e32ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:55.895710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a2aee-eb58-41af-b615-44b2511e32ee None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.domain.id=default, user.id=34ed7567d900480cb00efd1e75c2ddc5) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:10:55.896015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a2aee-eb58-41af-b615-44b2511e32ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:55.897539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a2aee-eb58-41af-b615-44b2511e32ee None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:55.905656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 30/56] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Aug 24 14:10:55 2022] GET /identity/v3/role_assignments?user.id=34ed7567d900480cb00efd1e75c2ddc5&scope.domain.id=default&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 530 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:10:55.944258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 27/57] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Aug 24 14:10:55 2022] POST /identity/v3/projects => generated 298 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:10:58.020340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1bb96-9cfa-4e46-bb75-f7cd9d7bd7dc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:58.020893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1bb96-9cfa-4e46-bb75-f7cd9d7bd7dc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:58.020893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1bb96-9cfa-4e46-bb75-f7cd9d7bd7dc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:58.022272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 31/58] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:10:58.028044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df76a3-a7ae-4a49-9ed8-4fd480ecd143 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:58.028512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df76a3-a7ae-4a49-9ed8-4fd480ecd143 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:58.028596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df76a3-a7ae-4a49-9ed8-4fd480ecd143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:58.048317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e2df76a3-a7ae-4a49-9ed8-4fd480ecd143 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:10:58.049556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e2df76a3-a7ae-4a49-9ed8-4fd480ecd143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:10:58.126443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 28/59] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:10:58.480178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1bb96-9cfa-4e46-bb75-f7cd9d7bd7dc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:58.599382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1bb96-9cfa-4e46-bb75-f7cd9d7bd7dc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:58.641148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08401a20-6b20-4f8b-9ae9-ce03580106c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:58.641731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08401a20-6b20-4f8b-9ae9-ce03580106c0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:58.642042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08401a20-6b20-4f8b-9ae9-ce03580106c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:58.642322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08401a20-6b20-4f8b-9ae9-ce03580106c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:58.643491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08401a20-6b20-4f8b-9ae9-ce03580106c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:58.644951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08401a20-6b20-4f8b-9ae9-ce03580106c0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:58.646587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 32/60] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:10:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:10:58.652302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df76a3-a7ae-4a49-9ed8-4fd480ecd143 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:10:58.740042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:10:58.760021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df76a3-a7ae-4a49-9ed8-4fd480ecd143 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:10:58.800543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30353365-6df4-48f6-b364-620029e8b7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:10:58.801186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353365-6df4-48f6-b364-620029e8b7cc None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:10:58.801468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353365-6df4-48f6-b364-620029e8b7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:10:58.801744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30353365-6df4-48f6-b364-620029e8b7cc None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:10:58.802457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30353365-6df4-48f6-b364-620029e8b7cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:10:58.804129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30353365-6df4-48f6-b364-620029e8b7cc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:10:58.965282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 29/61] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:10:58 2022] POST /identity/v3/users => generated 275 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:02.158352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc4c3-e371-4dac-badf-76d4a2c6a805 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:02.158352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc4c3-e371-4dac-badf-76d4a2c6a805 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:02.158352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc4c3-e371-4dac-badf-76d4a2c6a805 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:02.158352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 33/62] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:02.165892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873455-ec6c-4b23-aab4-77f6f76d731e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:02.166414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873455-ec6c-4b23-aab4-77f6f76d731e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:02.166860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873455-ec6c-4b23-aab4-77f6f76d731e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:02.207598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-03873455-ec6c-4b23-aab4-77f6f76d731e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:02.209747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-03873455-ec6c-4b23-aab4-77f6f76d731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:02.398811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 30/63] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:03.052832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc4c3-e371-4dac-badf-76d4a2c6a805 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:03.165820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:03.237753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922cae3-3a34-4e11-84ff-9266d8314873 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:03.239269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922cae3-3a34-4e11-84ff-9266d8314873 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:03.240053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922cae3-3a34-4e11-84ff-9266d8314873 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:03.242959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 31/64] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:03.252026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555579-55d4-4ceb-a2e4-a0ea0a5d63d2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:03.252951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555579-55d4-4ceb-a2e4-a0ea0a5d63d2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:03.253810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555579-55d4-4ceb-a2e4-a0ea0a5d63d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:03.288244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-34555579-55d4-4ceb-a2e4-a0ea0a5d63d2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:03.289847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-34555579-55d4-4ceb-a2e4-a0ea0a5d63d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:03.371176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc4c3-e371-4dac-badf-76d4a2c6a805 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:03.393587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 32/65] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:03.397293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:03.446261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:03.447199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:03.447632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:03.448089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:03.471806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:03.473490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39dff1-4b0a-47f9-9afc-f88abe989980 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:03.473946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39dff1-4b0a-47f9-9afc-f88abe989980 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:03.474412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39dff1-4b0a-47f9-9afc-f88abe989980 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:03.475979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 33/66] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:03.481371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:03.482489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:03.483396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:03.485678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 34/67] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:03.491619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742a4eb-45f4-4e7b-b14c-606849ed74a6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:03.492177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742a4eb-45f4-4e7b-b14c-606849ed74a6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:03.492656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742a4eb-45f4-4e7b-b14c-606849ed74a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:03.509111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:03.525457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8742a4eb-45f4-4e7b-b14c-606849ed74a6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:03.527672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8742a4eb-45f4-4e7b-b14c-606849ed74a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:03.661495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 34/68] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:03.701337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:03.706080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d3d3b-b9f5-4d5c-a6b5-ab0a98724f58 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:03.759318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:03.760615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:03.761214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:03.761700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:03.767649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:03.768227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:03.770845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:03.804344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 35/69] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:03.810424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742a4eb-45f4-4e7b-b14c-606849ed74a6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:03.961482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:04.049220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742a4eb-45f4-4e7b-b14c-606849ed74a6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:04.123860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:04.141673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:04.142701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.143051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.143436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] PATH_INFO: `/v3/users/34ed7567d900480cb00efd1e75c2ddc5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.150834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] RBAC: Authorizing `identity:get_user(user_id=34ed7567d900480cb00efd1e75c2ddc5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:04.152894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:04.155374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 35/70] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:03 2022] GET /identity/v3/users/34ed7567d900480cb00efd1e75c2ddc5 => generated 247 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:04.169957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:04.199528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d311a9b-32cf-4307-b549-e7db246174d7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:04.276166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:04.277238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.277721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.278242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.298780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:04.309002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:04.309941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:04.310753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:04.311376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 36/71] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:04.313542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:04.341655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454222f-0a8f-4c4e-a4fe-c239a6fb8fb5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:04.345602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:04.428597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca424b-2623-4ef9-9f8a-090eeeff2cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:04.429966 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca424b-2623-4ef9-9f8a-090eeeff2cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.430430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca424b-2623-4ef9-9f8a-090eeeff2cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.430828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca424b-2623-4ef9-9f8a-090eeeff2cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.481319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e64b7a-5a7a-4c72-ac0d-423e9c73349d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:04.506952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:04.516132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca424b-2623-4ef9-9f8a-090eeeff2cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:04.517621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca424b-2623-4ef9-9f8a-090eeeff2cc9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:04.554828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251fb0a-fdf8-422e-8a3f-cf10cea2b015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:04.555721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fb0a-fdf8-422e-8a3f-cf10cea2b015 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.556177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fb0a-fdf8-422e-8a3f-cf10cea2b015 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.556642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fb0a-fdf8-422e-8a3f-cf10cea2b015 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.573452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251fb0a-fdf8-422e-8a3f-cf10cea2b015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4251fb0a-fdf8-422e-8a3f-cf10cea2b015 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:04.584535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:04.585562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:04.586343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 37/72] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:04.587719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde8ffe-6f4e-4ac0-a291-1818984e9642 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.588009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde8ffe-6f4e-4ac0-a291-1818984e9642 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.588324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde8ffe-6f4e-4ac0-a291-1818984e9642 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.590790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 38/73] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:04.594199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde8ffe-6f4e-4ac0-a291-1818984e9642 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:04.644057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 36/74] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:04.645369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c39f5-5ca6-43bc-acf1-fd454d00a59b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.645659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c39f5-5ca6-43bc-acf1-fd454d00a59b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.645935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c39f5-5ca6-43bc-acf1-fd454d00a59b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.650654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 37/75] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:04.650654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c39f5-5ca6-43bc-acf1-fd454d00a59b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:04.708997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:04.742479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde8ffe-6f4e-4ac0-a291-1818984e9642 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:04.783684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:04.787452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec0e07-5372-4ac3-819a-02015dd18048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:04.788125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec0e07-5372-4ac3-819a-02015dd18048 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.788410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec0e07-5372-4ac3-819a-02015dd18048 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.788687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec0e07-5372-4ac3-819a-02015dd18048 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.789381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec0e07-5372-4ac3-819a-02015dd18048 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:04.789689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec0e07-5372-4ac3-819a-02015dd18048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:04.791263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec0e07-5372-4ac3-819a-02015dd18048 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:04.793815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c39f5-5ca6-43bc-acf1-fd454d00a59b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:04.801397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 39/76] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:04.802486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0b4f-f4cd-49a5-b8b2-58d9a171b917 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.802952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0b4f-f4cd-49a5-b8b2-58d9a171b917 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.803226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0b4f-f4cd-49a5-b8b2-58d9a171b917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.823959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a8ba0b4f-f4cd-49a5-b8b2-58d9a171b917 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:04.825204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a8ba0b4f-f4cd-49a5-b8b2-58d9a171b917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:04.852785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:04.853677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.854103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.854497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.855530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:04.855981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:04.858192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:04.874029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 38/77] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:04.875987 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:04.909338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 40/78] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:04.910698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f365d8-9d64-4d80-b4ca-b15fc17d600b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:04.911056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f365d8-9d64-4d80-b4ca-b15fc17d600b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:04.911605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f365d8-9d64-4d80-b4ca-b15fc17d600b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:04.932910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-07f365d8-9d64-4d80-b4ca-b15fc17d600b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:04.934237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-07f365d8-9d64-4d80-b4ca-b15fc17d600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:05.008024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70281d6d-4616-4874-b3bc-47c2d2823bca None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:05.018388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 41/79] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:05.019600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f365d8-9d64-4d80-b4ca-b15fc17d600b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:05.026019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:05.086614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:05.088270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:05.089208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:05.090028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:05.092679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:05.096579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:05.106817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 39/80] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:04 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:05.109210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:05.175038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:05.195640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f365d8-9d64-4d80-b4ca-b15fc17d600b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:05.268255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:05.269356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:05.269847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:05.270367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:05.292413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:05.293951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:05.296028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 42/81] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:05 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:05.299756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:05.316608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68002f-3220-43ac-b648-283573400a17 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:05.436350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:05.481098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92c635-1c49-4c48-81ae-49a61f847631 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:05.488958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162192df-05aa-455b-a82c-af50f661246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:05.489927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162192df-05aa-455b-a82c-af50f661246f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:05.490389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162192df-05aa-455b-a82c-af50f661246f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:05.490757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162192df-05aa-455b-a82c-af50f661246f None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:05.492293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162192df-05aa-455b-a82c-af50f661246f None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:05.494342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162192df-05aa-455b-a82c-af50f661246f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:05.496577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 40/82] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:05 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:05.497588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162192df-05aa-455b-a82c-af50f661246f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:05.526616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:05.527260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:05.527517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:05.527786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:05.528586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=34ed7567d900480cb00efd1e75c2ddc5) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:05.528827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:05.530303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:05.537491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 43/83] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:05 2022] GET /identity/v3/role_assignments?user.id=34ed7567d900480cb00efd1e75c2ddc5&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:05.538375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:05.639115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:05.651728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162192df-05aa-455b-a82c-af50f661246f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:05.699410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:05.700093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:05.700375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:05.700653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:05.709648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fed27-26fe-4b21-ad6b-883131e8d59f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:05.715214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:05.724411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:05.725620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:05.726720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 41/84] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:05.735698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:05.748366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:05.759941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:05.762821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:05.762821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:05.762821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:05.868137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:05.870504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:05.881562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f657-03a8-4f4b-b0f0-72f57d487e06 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:05.935182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:05.937142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:05.937597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:05.938092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:05.940197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:05.942586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:05.945686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 42/85] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:05.950103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:05.982668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:06.022160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 44/86] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:06.023234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:06.061104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:06.111112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2142e-b6b0-4524-bd3a-073a7a316a81 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:06.159582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:06.160375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:06.160789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:06.161484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:06.166816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4354988-986b-4cd0-a193-2b369758ce0d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:06.212376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:06.213006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:06.213288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:06.213586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:06.214331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:06.214731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:06.216095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:06.227662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 45/87] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:06.232857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:06.287470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:06.290572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:06.342051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:06.382139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:06.400602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97053acd-43c0-48e2-92ff-dbcc4446ce78 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:06.462222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:06.463352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:06.463793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:06.464293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:06.465642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:06.469425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:06.503170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 43/88] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:06.504688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:06.586024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 46/89] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:11:06 2022] POST /identity/v3/users => generated 289 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:06.586024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:06.739297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc9d9-0166-4d00-a0d3-a841bfe9ab12 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:06.791687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:06.817955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:06.818545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bc439-d60d-4b0e-8989-f602dfc05c74 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:06.819183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:06.819828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:06.820344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:06.823051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:06.824872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:06.827125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 44/90] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:06 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:06.828561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:06.900321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:06.902221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:06.903026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:06.903776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:06.907480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:06.911854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:06.916035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 47/91] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:06 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:06.918340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:06.956728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:07.044712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179fe4-41ec-4789-adf3-0d8e417818c4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:07.096945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9320b-4e9d-4015-8baf-b38bf27c0431 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:07.126201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:07.126965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:07.127312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:07.127654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:07.129009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:07.131008 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:07.133650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 45/92] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:06 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:07.142053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:07.175567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:07.177077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:07.177872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:07.178468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:07.179857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] RBAC: Adding query filter params (role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=7f091d543cf04f68b41e4456077bc426) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:07.181779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:07.184451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:07.201862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 48/93] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:06 2022] GET /identity/v3/role_assignments?user.id=7f091d543cf04f68b41e4456077bc426&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:07.204517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:07.221216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:07.348226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b077858-2206-40dd-81a7-76b0d5617847 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:07.362749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:07.411705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a9eb-be87-4676-80ea-a0acda92bbef None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:07.453776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:07.455434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:07.458200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:07.459140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:07.511756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:07.512409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:07.512687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:07.512964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:07.513722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=7f091d543cf04f68b41e4456077bc426) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:07.514075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:07.520467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:07.535576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 49/94] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:07 2022] GET /identity/v3/role_assignments?user.id=7f091d543cf04f68b41e4456077bc426&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=80a1f8f62542401a855036270bf2c51a => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:07.584755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:07.586330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:07.659418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:07.729965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 46/95] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:07.740546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:07.926926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6a903-043f-4c08-9ec6-c66ff2ed66e2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:08.020985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f808a03-c86c-41dc-aab9-49112fb2f871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:08.022125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808a03-c86c-41dc-aab9-49112fb2f871 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:08.022622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808a03-c86c-41dc-aab9-49112fb2f871 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:08.023112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808a03-c86c-41dc-aab9-49112fb2f871 None admin] PATH_INFO: `/v3/projects/6de9ba88c5ff427caec1957e1305c2f6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:08.025487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f808a03-c86c-41dc-aab9-49112fb2f871 None admin] RBAC: Authorizing `identity:get_project(project_id=6de9ba88c5ff427caec1957e1305c2f6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:08.028364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f808a03-c86c-41dc-aab9-49112fb2f871 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:08.032768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 50/96] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:07 2022] GET /identity/v3/projects/6de9ba88c5ff427caec1957e1305c2f6 => generated 312 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:08.042596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:08.273100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740465-aaec-4e2d-93b5-1c3e4cebd9fa None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:08.356223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35506d7a-b7f3-4f6e-85c8-1dccef87aeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:08.357258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35506d7a-b7f3-4f6e-85c8-1dccef87aeae None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:08.357751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35506d7a-b7f3-4f6e-85c8-1dccef87aeae None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:08.362426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35506d7a-b7f3-4f6e-85c8-1dccef87aeae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:08.363962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35506d7a-b7f3-4f6e-85c8-1dccef87aeae None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=6de9ba88c5ff427caec1957e1305c2f6, user.id=7f091d543cf04f68b41e4456077bc426) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:08.364515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35506d7a-b7f3-4f6e-85c8-1dccef87aeae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:08.374591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35506d7a-b7f3-4f6e-85c8-1dccef87aeae None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:08.379181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 47/97] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:08 2022] GET /identity/v3/role_assignments?user.id=7f091d543cf04f68b41e4456077bc426&scope.project.id=6de9ba88c5ff427caec1957e1305c2f6&role.id=80a1f8f62542401a855036270bf2c51a => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:08.418668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a68efa-ff87-41eb-baa4-905757d360c1 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:08.418668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a68efa-ff87-41eb-baa4-905757d360c1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:08.418668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a68efa-ff87-41eb-baa4-905757d360c1 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:08.424715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 51/98] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:08 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:08.442446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01643-6a25-4a91-8680-bd0ba9993056 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:08.442990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01643-6a25-4a91-8680-bd0ba9993056 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:08.443419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01643-6a25-4a91-8680-bd0ba9993056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:08.465320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-55a01643-6a25-4a91-8680-bd0ba9993056 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:08.466842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-55a01643-6a25-4a91-8680-bd0ba9993056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:08.488660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:08.567460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 48/99] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:08.955890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a68efa-ff87-41eb-baa4-905757d360c1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:08.969841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:09.200815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a68efa-ff87-41eb-baa4-905757d360c1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:09.266269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:09.267277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:09.267753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:09.268243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:09.289331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:09.299746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:09.301193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:09.303424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 52/100] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:11:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:09.309651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a01643-6a25-4a91-8680-bd0ba9993056 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:09.473882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a01643-6a25-4a91-8680-bd0ba9993056 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:09.520425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de80faa-5cd9-429e-bd29-363632e6701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:09.520961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de80faa-5cd9-429e-bd29-363632e6701c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:09.521243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de80faa-5cd9-429e-bd29-363632e6701c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:09.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de80faa-5cd9-429e-bd29-363632e6701c None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:09.523734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de80faa-5cd9-429e-bd29-363632e6701c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:09.524076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de80faa-5cd9-429e-bd29-363632e6701c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:09.525525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de80faa-5cd9-429e-bd29-363632e6701c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:09.536526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 49/101] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:09.543803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:09.697855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:09.770946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70178281-0822-4eb3-bd6e-95460113f9e5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:09.797054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfecdfd-b1e4-40b7-99c5-24262c71679b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:09.797333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfecdfd-b1e4-40b7-99c5-24262c71679b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:09.797627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfecdfd-b1e4-40b7-99c5-24262c71679b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:09.799039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 50/102] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:09.806010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a220-c5b9-4caf-97b2-3de9b1c4e6bb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:09.806289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a220-c5b9-4caf-97b2-3de9b1c4e6bb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:09.806596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a220-c5b9-4caf-97b2-3de9b1c4e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:09.830034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-af32a220-c5b9-4caf-97b2-3de9b1c4e6bb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:09.832096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-af32a220-c5b9-4caf-97b2-3de9b1c4e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:09.846624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:09.894639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:09.895962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:09.896343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:09.896855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] PATH_INFO: `/v3/users/34ed7567d900480cb00efd1e75c2ddc5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:09.899054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] RBAC: Authorizing `identity:get_user(user_id=34ed7567d900480cb00efd1e75c2ddc5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:09.901807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:09.913610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 53/103] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:09 2022] GET /identity/v3/users/34ed7567d900480cb00efd1e75c2ddc5 => generated 247 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:09.925257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:09.926798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 51/104] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:10.082420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99816b91-4f72-4653-ac89-bbe08fac7607 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:10.133157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:10.134278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:10.134709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:10.135133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:10.137173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:10.140292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:10.143814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 54/105] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:09 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:10.153148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a220-c5b9-4caf-97b2-3de9b1c4e6bb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:10.333317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a220-c5b9-4caf-97b2-3de9b1c4e6bb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:10.403926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:10.414275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:10.415571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:10.416097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:10.416587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/34ed7567d900480cb00efd1e75c2ddc5/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:10.419996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:10.421748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=aefb95afbe7845b5a5da1d55abf22a62, user_id=34ed7567d900480cb00efd1e75c2ddc5, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:10.423683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:10.528367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 52/106] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:10 2022] PUT /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/34ed7567d900480cb00efd1e75c2ddc5/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:10.692864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cf438-afc4-4884-ba89-8133749a9433 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:10.780286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:10.781378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:10.781867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:10.782372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:10.814033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:10.824636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:10.826328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:10.827416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 55/107] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:10.835932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:10.847929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:11.087067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d957ce-056e-4683-807a-1a1fe9e674e1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:11.178142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:11.179189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:11.179677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:11.180164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:11.181422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:11.181953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:11.184585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:11.209152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 53/108] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:11.219677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:11.375805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:11.470942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b22f33-cb3a-406d-96d0-3ebc32a098d6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:11.554661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649010-4cac-436e-a98f-57ce436adae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:11.555726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649010-4cac-436e-a98f-57ce436adae7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:11.556214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649010-4cac-436e-a98f-57ce436adae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:11.556707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649010-4cac-436e-a98f-57ce436adae7 None admin] PATH_INFO: `/v3/users/3373df2ccc7a441a82be8841bd149de4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:11.588214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649010-4cac-436e-a98f-57ce436adae7 None admin] RBAC: Authorizing `identity:get_user(user_id=3373df2ccc7a441a82be8841bd149de4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:11.591026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649010-4cac-436e-a98f-57ce436adae7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:11.594150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 56/109] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:11 2022] GET /identity/v3/users/3373df2ccc7a441a82be8841bd149de4 => generated 289 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:11.603882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:11.613874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83f57d-d80b-434a-85be-7a2a0a7a59b4 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:11.614356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83f57d-d80b-434a-85be-7a2a0a7a59b4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:11.614832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83f57d-d80b-434a-85be-7a2a0a7a59b4 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:11.617741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 57/110] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:11.624079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d69f32-e26f-4479-89ca-818ad170423f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:11.624495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d69f32-e26f-4479-89ca-818ad170423f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:11.624999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d69f32-e26f-4479-89ca-818ad170423f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:11.672368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b9d69f32-e26f-4479-89ca-818ad170423f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:11.674416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b9d69f32-e26f-4479-89ca-818ad170423f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:11.775224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:11.859455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a6447-5cc8-4599-b90a-2df61d96603b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:11.924332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 58/111] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:11.956164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143b6f-0067-4a7f-b434-bb94eab9063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:11.956822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143b6f-0067-4a7f-b434-bb94eab9063a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:11.957105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143b6f-0067-4a7f-b434-bb94eab9063a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:11.957380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143b6f-0067-4a7f-b434-bb94eab9063a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:12.016762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdef15-9ef5-45bd-bb72-4cd05c44c434 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:12.017195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdef15-9ef5-45bd-bb72-4cd05c44c434 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:12.017677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdef15-9ef5-45bd-bb72-4cd05c44c434 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:12.020542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 59/112] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:12.032188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31419d3-21dd-49f8-8eac-0c030bb59556 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:12.032640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31419d3-21dd-49f8-8eac-0c030bb59556 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:12.033103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31419d3-21dd-49f8-8eac-0c030bb59556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:12.096384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b31419d3-21dd-49f8-8eac-0c030bb59556 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:12.098878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b31419d3-21dd-49f8-8eac-0c030bb59556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:12.100994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60143b6f-0067-4a7f-b434-bb94eab9063a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:12.103765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60143b6f-0067-4a7f-b434-bb94eab9063a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:12.272924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:12.318514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:12.346191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 60/113] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:12.419732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 54/114] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:12.429314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31419d3-21dd-49f8-8eac-0c030bb59556 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:12.639090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674e47-5681-4887-bdd4-e035ed4704fb None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:12.639932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674e47-5681-4887-bdd4-e035ed4704fb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:12.639932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674e47-5681-4887-bdd4-e035ed4704fb None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:12.641676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 55/115] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:12.651256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00e869-c947-4446-b711-69a28d118efe None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:12.651645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00e869-c947-4446-b711-69a28d118efe None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:12.651829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00e869-c947-4446-b711-69a28d118efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:12.690705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7a00e869-c947-4446-b711-69a28d118efe None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:12.692027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7a00e869-c947-4446-b711-69a28d118efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:12.713653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31419d3-21dd-49f8-8eac-0c030bb59556 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:12.815337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:12.816007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:12.816291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:12.816554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:12.819237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:12.822471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:12.826215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 61/116] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:12 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:12.828926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:12.846914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 56/117] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:12.847547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00e869-c947-4446-b711-69a28d118efe None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:12.954630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:13.004181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:13.016949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00e869-c947-4446-b711-69a28d118efe None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:13.084737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f6d46-f3f7-4bd6-96f2-e879c7e5d971 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:13.088253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:13.089294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.089743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.090226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:13.092199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:13.095531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:13.098534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 57/118] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:13.106025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:13.204179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:13.205323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.205761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.206274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:13.253355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:13.266889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:13.267814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:13.268567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:13.270019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 62/119] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:13.272106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:13.382960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16ab5-a32d-46fc-ad9c-2fc0efaee854 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:13.437008 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:13.437843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.438194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.438477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:13.439241 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] RBAC: Adding query filter params (role.id=64e702b1f13d4b62988575b426f8f8a3, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=3373df2ccc7a441a82be8841bd149de4) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:13.439557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:13.441051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:13.448105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 58/120] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:13 2022] GET /identity/v3/role_assignments?user.id=3373df2ccc7a441a82be8841bd149de4&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=64e702b1f13d4b62988575b426f8f8a3 => generated 259 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:13.448992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:13.472115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba055e-7861-4301-ade1-b2b002f4d8b0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:13.508108 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:13.516813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:13.561073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:13.563258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.563624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.563910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:13.564538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:13.566139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:13.653714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b70bc6-578d-4285-b087-4059cb4391fe None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:13.668423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 63/121] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:11:13 2022] POST /identity/v3/projects => generated 302 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:13.668423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:13.752950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46201a-cc81-4127-97a3-12fb49f94c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:13.754458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46201a-cc81-4127-97a3-12fb49f94c27 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.755036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46201a-cc81-4127-97a3-12fb49f94c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.755611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46201a-cc81-4127-97a3-12fb49f94c27 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:13.757022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46201a-cc81-4127-97a3-12fb49f94c27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:13.757914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46201a-cc81-4127-97a3-12fb49f94c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:13.762480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46201a-cc81-4127-97a3-12fb49f94c27 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:13.783249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 59/122] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:13.785715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563145c-675b-42e1-b72c-9cae9049159c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.786514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563145c-675b-42e1-b72c-9cae9049159c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.787017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563145c-675b-42e1-b72c-9cae9049159c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:13.789576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 60/123] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:13.792433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195ea7-d474-44ac-96ff-87889dbd28e8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.792935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195ea7-d474-44ac-96ff-87889dbd28e8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.794439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195ea7-d474-44ac-96ff-87889dbd28e8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:13.798519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 61/124] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:13.803400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195ea7-d474-44ac-96ff-87889dbd28e8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:13.922589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb09a-11da-4eca-9470-bfe31ad1a688 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:13.993437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:13.995709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c8852-f1e3-490a-aac2-efde10d92d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:13.996459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c8852-f1e3-490a-aac2-efde10d92d97 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:13.996459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c8852-f1e3-490a-aac2-efde10d92d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:13.996459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c8852-f1e3-490a-aac2-efde10d92d97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.016144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c8852-f1e3-490a-aac2-efde10d92d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-075c8852-f1e3-490a-aac2-efde10d92d97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:14.016519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:14.018039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 64/125] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:14.020300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa0b62-b8e2-4309-9bc5-f2df9f333388 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.021883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa0b62-b8e2-4309-9bc5-f2df9f333388 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.021883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa0b62-b8e2-4309-9bc5-f2df9f333388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.033115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195ea7-d474-44ac-96ff-87889dbd28e8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:14.042683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-51aa0b62-b8e2-4309-9bc5-f2df9f333388 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:14.043973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-51aa0b62-b8e2-4309-9bc5-f2df9f333388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:14.068564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:14.106292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d693173-7851-4e32-a487-19e0acfb9cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:14.107937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d693173-7851-4e32-a487-19e0acfb9cde None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.107937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d693173-7851-4e32-a487-19e0acfb9cde None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.107937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d693173-7851-4e32-a487-19e0acfb9cde None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.109740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d693173-7851-4e32-a487-19e0acfb9cde None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:14.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d693173-7851-4e32-a487-19e0acfb9cde None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:14.118190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 62/126] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:13 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:14.118190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b21a-5d53-4190-a7af-89aae0ea9a92 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.118190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b21a-5d53-4190-a7af-89aae0ea9a92 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.118190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877b21a-5d53-4190-a7af-89aae0ea9a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.139749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0877b21a-5d53-4190-a7af-89aae0ea9a92 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:14.141106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0877b21a-5d53-4190-a7af-89aae0ea9a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:14.201495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 65/127] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:14.202863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa0b62-b8e2-4309-9bc5-f2df9f333388 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:14.265396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 63/128] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:14.266737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877b21a-5d53-4190-a7af-89aae0ea9a92 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:14.363546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa0b62-b8e2-4309-9bc5-f2df9f333388 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:14.415729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:14.416530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.416912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.417290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.418156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:14.418594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:14.420301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:14.431566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 66/129] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:14.437853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:14.449083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:14.461257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:14.484301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877b21a-5d53-4190-a7af-89aae0ea9a92 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:14.559016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:14.559561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.559851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.560121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.567655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:14.568479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:14.570305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 64/130] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:14 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:14.576432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:14.632178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c2920-1185-4383-8a4f-e608b293db67 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:14.683413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:14.684461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.684990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.685466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.687679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:14.689717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:14.692114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 67/131] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:14 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:14.698345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:14.826483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:14.829778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff402779-9909-437e-98b7-138ed4bb3566 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:14.863695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb0da-e34f-413a-a472-814183868a92 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:14.908329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:14.909070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.909357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.909634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/7f091d543cf04f68b41e4456077bc426/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.914414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] RBAC: Authorizing `identity:create_grant(project_id=aefb95afbe7845b5a5da1d55abf22a62, user_id=7f091d543cf04f68b41e4456077bc426, role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:14.915903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:14.935725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:14.936563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:14.940314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:14.940314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:14.940314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:14.940314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:14.942123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 68/132] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:14 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:14.942761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:15.022811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 65/133] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:14 2022] PUT /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/7f091d543cf04f68b41e4456077bc426/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:11:15.024257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:15.077556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:15.140787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75f4bc-3573-4273-b85b-9282f241eaf9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:15.215868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b2e08-0e30-4703-a048-1e680ecd6411 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:15.223366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:15.224374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:15.224855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:15.225329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:15.247062 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:15.252720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:15.253731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:15.254625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 69/134] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:15.255285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:15.272851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:15.273910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:15.274500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:15.274942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:15.275432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:15.291852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:15.297403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:15.298467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:15.299376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:15.301462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 66/135] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:11:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:15.302524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:15.387040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a62b4b-c83c-4139-bb84-90cd1d66245e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:15.428867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:15.429568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:15.429854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:15.430173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/7f091d543cf04f68b41e4456077bc426/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:15.432615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] RBAC: Authorizing `identity:create_grant(project_id=aefb95afbe7845b5a5da1d55abf22a62, user_id=7f091d543cf04f68b41e4456077bc426, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:15.434414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:15.438997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:15.488119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873140ab-469c-42e7-b07a-c9f15d4a885d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:15.522764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 70/136] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:15 2022] PUT /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/7f091d543cf04f68b41e4456077bc426/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:15.524041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:15.585538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:15.590103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:15.590103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:15.590103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:15.590103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:15.590103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:15.592115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:15.620900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 67/137] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:15.642401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:15.654650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:15.684357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edb600-7d3e-45cb-9d78-922d1bef75ea None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:15.732617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:15.734854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:15.735646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:15.737064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:15.738009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:15.739370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:15.743710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:15.764595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 71/138] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:15.770274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:15.919927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9a7c1-cc7e-4176-afff-0b1638180b7a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:15.945456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd765e9-a950-472f-93cc-e9ecde4da8fa None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:15.986339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:15.987060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:15.988141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:15.988643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:15.989152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:15.991261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:15.992689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:15.993431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:15.993782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:15.994016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:15.994227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] PATH_INFO: `/v3/users/34ed7567d900480cb00efd1e75c2ddc5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:15.995630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] RBAC: Authorizing `identity:get_user(user_id=34ed7567d900480cb00efd1e75c2ddc5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:15.997191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 68/139] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:15 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:15.997535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:16.000526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 72/140] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:15 2022] GET /identity/v3/users/34ed7567d900480cb00efd1e75c2ddc5 => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:16.007275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:16.009937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:16.020656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:16.196170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413d82c-d9ce-4820-9302-a198d511d054 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:16.221572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5afb-7567-423c-901f-31bf4fe0b259 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:16.238596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808277-b709-427c-a7db-88569124e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:16.244249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808277-b709-427c-a7db-88569124e893 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:16.244249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808277-b709-427c-a7db-88569124e893 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:16.244249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808277-b709-427c-a7db-88569124e893 None admin] PATH_INFO: `/v3/projects/6de9ba88c5ff427caec1957e1305c2f6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:16.244249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea808277-b709-427c-a7db-88569124e893 None admin] RBAC: Authorizing `identity:get_project(project_id=6de9ba88c5ff427caec1957e1305c2f6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:16.244249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea808277-b709-427c-a7db-88569124e893 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:16.245311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 73/141] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:16 2022] GET /identity/v3/projects/6de9ba88c5ff427caec1957e1305c2f6 => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:16.246659 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907c79b-7d4d-4f36-abc2-b10c9f8b9d1e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:16.246936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907c79b-7d4d-4f36-abc2-b10c9f8b9d1e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:16.247204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907c79b-7d4d-4f36-abc2-b10c9f8b9d1e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:16.249650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 74/142] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:16.258651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f4ce1-1403-42f6-aefe-7ade9b867ecc None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:16.258930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f4ce1-1403-42f6-aefe-7ade9b867ecc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:16.259296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f4ce1-1403-42f6-aefe-7ade9b867ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:16.284950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:16.286136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:16.286549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:16.286949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:16.302796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ff7f4ce1-1403-42f6-aefe-7ade9b867ecc None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:16.305130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ff7f4ce1-1403-42f6-aefe-7ade9b867ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:16.421027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:16.428523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 75/143] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:16.429828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f4ce1-1403-42f6-aefe-7ade9b867ecc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:16.487359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:16.488556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:16.506047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:16.607089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f4ce1-1403-42f6-aefe-7ade9b867ecc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:16.652855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:16.653919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:16.654397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:16.654821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] PATH_INFO: `/v3/projects/6de9ba88c5ff427caec1957e1305c2f6/users/7f091d543cf04f68b41e4456077bc426/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:16.658047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] RBAC: Authorizing `identity:create_grant(project_id=6de9ba88c5ff427caec1957e1305c2f6, user_id=7f091d543cf04f68b41e4456077bc426, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:16.659786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:16.724096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 69/144] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:16.730285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:16.805103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 76/145] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:16 2022] PUT /identity/v3/projects/6de9ba88c5ff427caec1957e1305c2f6/users/7f091d543cf04f68b41e4456077bc426/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:16.887623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2614591-de6f-4254-8b00-5fa9d0a700e8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:16.898221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:16.933821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:16.935388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:16.935816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:16.936302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:16.938121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:16.940034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:16.942099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 70/146] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:16 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:16.952438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:17.074918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:17.158338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:17.169560 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c45520-d4f4-4d8f-b1b4-662e86304e9f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:17.246281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:17.247553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:17.248034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:17.248512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:17.250946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=34ed7567d900480cb00efd1e75c2ddc5) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:17.251351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:17.254214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:17.270330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 77/147] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:16 2022] GET /identity/v3/role_assignments?user.id=34ed7567d900480cb00efd1e75c2ddc5&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:17.426515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8baa13-c145-44be-973e-9eb775a6a8bd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:17.507274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42658e6b-41ae-43b7-95e0-e1f9c1d13d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:17.508387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658e6b-41ae-43b7-95e0-e1f9c1d13d56 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:17.508885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658e6b-41ae-43b7-95e0-e1f9c1d13d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:17.509367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42658e6b-41ae-43b7-95e0-e1f9c1d13d56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:17.537654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42658e6b-41ae-43b7-95e0-e1f9c1d13d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-42658e6b-41ae-43b7-95e0-e1f9c1d13d56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:17.551656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:17.552606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:17.553380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:17.553956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 71/148] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:17.564375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:17.693094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc04c1d-f183-4565-8959-a59ead62d70c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:17.693482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc04c1d-f183-4565-8959-a59ead62d70c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:17.694025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc04c1d-f183-4565-8959-a59ead62d70c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:17.696647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 72/149] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:17.716264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814cfb-aa65-4bbf-a945-5ffd30d79bc5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:17.716692 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814cfb-aa65-4bbf-a945-5ffd30d79bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:17.717032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814cfb-aa65-4bbf-a945-5ffd30d79bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:17.770161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fc814cfb-aa65-4bbf-a945-5ffd30d79bc5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:17.772242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fc814cfb-aa65-4bbf-a945-5ffd30d79bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:17.852634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:17.871897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4e6ac-34b5-4126-bb68-bf6922ca618e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:17.928214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:17.950249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 73/150] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:17.958651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98113c3e-0fa5-4482-8ea1-51b87e964f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:17.959679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98113c3e-0fa5-4482-8ea1-51b87e964f81 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:17.960170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98113c3e-0fa5-4482-8ea1-51b87e964f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:17.960766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98113c3e-0fa5-4482-8ea1-51b87e964f81 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:17.961671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98113c3e-0fa5-4482-8ea1-51b87e964f81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:17.963300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98113c3e-0fa5-4482-8ea1-51b87e964f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:17.965081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98113c3e-0fa5-4482-8ea1-51b87e964f81 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:17.977871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 78/151] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:17.987839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814cfb-aa65-4bbf-a945-5ffd30d79bc5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:18.081077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1ead-1e2a-4052-a0bd-917533ca1a4b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:18.081710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1ead-1e2a-4052-a0bd-917533ca1a4b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:18.082233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f1ead-1e2a-4052-a0bd-917533ca1a4b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:18.083894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 79/152] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:18.090229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d45c7-6a7e-4e4e-8745-9f32cfd511ab None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:18.090841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d45c7-6a7e-4e4e-8745-9f32cfd511ab None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:18.091311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d45c7-6a7e-4e4e-8745-9f32cfd511ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:18.112364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b74d45c7-6a7e-4e4e-8745-9f32cfd511ab None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:18.114020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b74d45c7-6a7e-4e4e-8745-9f32cfd511ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:18.185778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814cfb-aa65-4bbf-a945-5ffd30d79bc5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:18.219888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 80/153] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:18.241588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:18.242719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:18.243219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:18.243711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] PATH_INFO: `/v3/users/3373df2ccc7a441a82be8841bd149de4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:18.245396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] RBAC: Authorizing `identity:get_user(user_id=3373df2ccc7a441a82be8841bd149de4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:18.247244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:18.249983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 74/154] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:17 2022] GET /identity/v3/users/3373df2ccc7a441a82be8841bd149de4 => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:18.259705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d45c7-6a7e-4e4e-8745-9f32cfd511ab None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:18.488186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:18.490542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d45c7-6a7e-4e4e-8745-9f32cfd511ab None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:18.523983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:18.573337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:18.574459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:18.574964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:18.575440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:18.579813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:18.582733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:18.586291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 81/155] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:18 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:18.593174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:18.600005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:18.744880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07972dd1-b2b4-446c-b228-7eb76332aea4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:18.788578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:18.789437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:18.789720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:18.790027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:18.791393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:18.792877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:18.795314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 75/156] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:18.796881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:18.813368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67430e-475f-4d63-b4c3-6d911c83daf1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:18.879815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:18.881104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:18.881582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:18.882117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/3373df2ccc7a441a82be8841bd149de4/roles/64e702b1f13d4b62988575b426f8f8a3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:18.885972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] RBAC: Authorizing `identity:create_grant(project_id=aefb95afbe7845b5a5da1d55abf22a62, user_id=3373df2ccc7a441a82be8841bd149de4, role_id=64e702b1f13d4b62988575b426f8f8a3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:18.888898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:18.922850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:18.926645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c4406-a756-4d95-a7c5-3ad53d7a474c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:18.969498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 82/157] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:18 2022] PUT /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/users/3373df2ccc7a441a82be8841bd149de4/roles/64e702b1f13d4b62988575b426f8f8a3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:18.970384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:18.981736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:18.985448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:18.985958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:18.986541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:19.018164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:19.032313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:19.034665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:19.037056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 76/158] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:19.044211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:19.112587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:19.262648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171d4d-6bc9-4c22-8f3f-bc45bf7b1be3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:19.282810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c786ac-5434-4d6d-b523-32199a75f294 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:19.317115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97504cbe-fc31-4532-8870-ea69494b83aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:19.318079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97504cbe-fc31-4532-8870-ea69494b83aa None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:19.318370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97504cbe-fc31-4532-8870-ea69494b83aa None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:19.318638 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97504cbe-fc31-4532-8870-ea69494b83aa None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:19.319843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97504cbe-fc31-4532-8870-ea69494b83aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:19.321698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97504cbe-fc31-4532-8870-ea69494b83aa None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:19.331383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7528b0-f123-4049-89ee-f08777e2ee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:19.332896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7528b0-f123-4049-89ee-f08777e2ee9e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:19.333652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7528b0-f123-4049-89ee-f08777e2ee9e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:19.334456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7528b0-f123-4049-89ee-f08777e2ee9e None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:19.336039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7528b0-f123-4049-89ee-f08777e2ee9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:19.336961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7528b0-f123-4049-89ee-f08777e2ee9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:19.339955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7528b0-f123-4049-89ee-f08777e2ee9e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:19.352023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 77/159] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:19.356124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4b846-ce13-441c-ad07-cab51ef4f6a2 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:19.357162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4b846-ce13-441c-ad07-cab51ef4f6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:19.357544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4b846-ce13-441c-ad07-cab51ef4f6a2 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:19.361402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 78/160] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:19.362551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4b846-ce13-441c-ad07-cab51ef4f6a2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:19.394900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:19.499425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4b846-ce13-441c-ad07-cab51ef4f6a2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:19.525675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 83/161] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:11:18 2022] POST /identity/v3/users => generated 283 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:19.537598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb7a4b-163d-489e-b066-b47934bdfb44 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:19.537598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb7a4b-163d-489e-b066-b47934bdfb44 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:19.537940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb7a4b-163d-489e-b066-b47934bdfb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:19.603553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:19.606527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:19.607065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:19.607317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-adbb7a4b-163d-489e-b066-b47934bdfb44 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:19.608886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:19.612989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-adbb7a4b-163d-489e-b066-b47934bdfb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:19.613576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:19.620648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:19.630730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 79/162] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:19 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:19.636732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:19.801540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 84/163] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:19.845501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:19.853853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff38989-b80c-4c84-bfcf-e0f618c51588 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:19.899373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:19.900009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:19.900290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:19.900566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:19.986412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:19.987935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:20.107195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:20.121015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 80/164] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:20.131278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb7a4b-163d-489e-b066-b47934bdfb44 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:20.166039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:20.210904 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:20.292654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb7a4b-163d-489e-b066-b47934bdfb44 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:20.376546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:20.377653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:20.378164 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:20.378670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:20.386033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:20.386033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:20.386641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 85/165] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:20 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:20.395441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:20.452895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087c2ed-ad31-436d-81d3-70fb911c6fd5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:20.544045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19817e51-4635-4e5c-af94-addf53c7b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:20.545098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817e51-4635-4e5c-af94-addf53c7b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:20.545581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817e51-4635-4e5c-af94-addf53c7b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:20.546153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817e51-4635-4e5c-af94-addf53c7b4a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:20.571816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:20.580692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c5352-a0d1-454a-a0de-cfdf0d356f0b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:20.582458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19817e51-4635-4e5c-af94-addf53c7b4a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-19817e51-4635-4e5c-af94-addf53c7b4a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:20.592376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:20.594327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 81/166] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:20.601429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cfdbb-9220-4c12-899b-c54eaa507068 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:20.603460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cfdbb-9220-4c12-899b-c54eaa507068 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:20.603460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cfdbb-9220-4c12-899b-c54eaa507068 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:20.603460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 82/167] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:20.604907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cfdbb-9220-4c12-899b-c54eaa507068 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:20.643180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250079a-aeb0-42e4-8d48-30a9482123ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:20.644135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250079a-aeb0-42e4-8d48-30a9482123ec None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:20.644600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250079a-aeb0-42e4-8d48-30a9482123ec None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:20.645056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250079a-aeb0-42e4-8d48-30a9482123ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:20.646149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250079a-aeb0-42e4-8d48-30a9482123ec None admin] RBAC: Adding query filter params (role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=7f091d543cf04f68b41e4456077bc426) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:20.646690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250079a-aeb0-42e4-8d48-30a9482123ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:20.648381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250079a-aeb0-42e4-8d48-30a9482123ec None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:20.656418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 86/168] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:20 2022] GET /identity/v3/role_assignments?user.id=7f091d543cf04f68b41e4456077bc426&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:20.662997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99de31-8710-45aa-b8b9-18f6a7b04542 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:20.663487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99de31-8710-45aa-b8b9-18f6a7b04542 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:20.663969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99de31-8710-45aa-b8b9-18f6a7b04542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:20.695796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ec99de31-8710-45aa-b8b9-18f6a7b04542 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:20.698200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ec99de31-8710-45aa-b8b9-18f6a7b04542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:20.724084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:20.845541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 87/169] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:20.856189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cfdbb-9220-4c12-899b-c54eaa507068 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:20.906459 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:20.907904 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:20.908512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:20.909034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:20.910067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:20.910587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:20.912526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:20.925264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 83/170] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:20.932652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99de31-8710-45aa-b8b9-18f6a7b04542 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:21.071697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:21.087742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99de31-8710-45aa-b8b9-18f6a7b04542 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:21.135830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:21.137417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:21.137903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:21.138486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:21.141031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:21.142815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:21.145314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 88/171] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:20 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:21.151187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:21.303685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425ffa2-ecc5-454b-a27d-87d48f13f9aa None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:21.320983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:21.347386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:21.348000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:21.348280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:21.348457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:21.349111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:21.490495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:21.492749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:21.504124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08f5c-2fbd-4373-bed6-afc8d9eb8945 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:21.559808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:21.561490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:21.562113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:21.563499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:21.587577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:21.598100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:21.599910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:21.602395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 89/172] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:21.610818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:21.650187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:21.701276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 84/173] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 14:11:21.716679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:21.736345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:21.942594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8606ac-edbb-416c-bb7d-d710555496a2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:21.949490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b740-f4f5-4605-8fdb-d661623edc69 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:22.026714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:22.027863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:22.028376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:22.028872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:22.031398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:22.034520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:22.038635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 85/174] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:21 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:22.049702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:22.074723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:22.075909 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:22.076413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:22.077476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:22.079434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:22.079974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:22.087648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:22.115097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 90/175] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 506 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:22.125961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:22.307550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:22.350717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b47c2-19f2-4d2d-b9fe-82d9a2699b5b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:22.382092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:22.451448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a3b2d-dacf-401b-ae9b-fc3d970a2a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:22.452478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a3b2d-dacf-401b-ae9b-fc3d970a2a31 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:22.452939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a3b2d-dacf-401b-ae9b-fc3d970a2a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:22.453224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a3b2d-dacf-401b-ae9b-fc3d970a2a31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:22.453730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dbaf9-c5d8-47c0-b1f5-5a428de28ca0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:22.458454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a3b2d-dacf-401b-ae9b-fc3d970a2a31 None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=7f091d543cf04f68b41e4456077bc426) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:22.458889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a3b2d-dacf-401b-ae9b-fc3d970a2a31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:22.460521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a3b2d-dacf-401b-ae9b-fc3d970a2a31 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:22.475561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 86/176] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:22 2022] GET /identity/v3/role_assignments?user.id=7f091d543cf04f68b41e4456077bc426&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=80a1f8f62542401a855036270bf2c51a => generated 608 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:22.495760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e767aa-7ecf-4047-bd1b-20cfd88e88d3 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:22.496116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e767aa-7ecf-4047-bd1b-20cfd88e88d3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:22.496368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e767aa-7ecf-4047-bd1b-20cfd88e88d3 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:22.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 87/177] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:22.508273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be1d2-c509-4758-9d03-c7114268e199 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:22.508781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be1d2-c509-4758-9d03-c7114268e199 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:22.509363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be1d2-c509-4758-9d03-c7114268e199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:22.585160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-346be1d2-c509-4758-9d03-c7114268e199 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:22.587261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-346be1d2-c509-4758-9d03-c7114268e199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:22.602026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:22.602026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:22.602026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:22.602026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] PATH_INFO: `/v3/users/7f091d543cf04f68b41e4456077bc426` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:22.602450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] RBAC: Authorizing `identity:get_user(user_id=7f091d543cf04f68b41e4456077bc426)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:22.604747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:22.608609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 91/178] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:22 2022] GET /identity/v3/users/7f091d543cf04f68b41e4456077bc426 => generated 275 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:22.617700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:22.856355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 88/179] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:22.895599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520175e3-6bcc-4a9e-b881-330d81d6dfe1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:22.969329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:22.970581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:22.971206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:22.971786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:23.007696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:23.111552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:23.114602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:23.353559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be1d2-c509-4758-9d03-c7114268e199 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:23.373116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 92/180] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:23.383202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:23.492843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:23.593811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608360e-22d7-4ee3-99a4-606a4fc12476 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:23.662902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:23.666337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:23.667980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:23.668697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:23.669389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] PATH_INFO: `/v3/projects/6de9ba88c5ff427caec1957e1305c2f6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:23.673633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] RBAC: Authorizing `identity:get_project(project_id=6de9ba88c5ff427caec1957e1305c2f6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:23.676892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:23.680484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 93/181] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:23 2022] GET /identity/v3/projects/6de9ba88c5ff427caec1957e1305c2f6 => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:23.690659 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:23.767115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be1d2-c509-4758-9d03-c7114268e199 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:23.908573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298022ab-b0e0-4eb1-9973-a094a805f96a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:23.940360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6c99a-80ce-4173-a946-690b7c084445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:23.941419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c99a-80ce-4173-a946-690b7c084445 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:23.942141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c99a-80ce-4173-a946-690b7c084445 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:23.942767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c99a-80ce-4173-a946-690b7c084445 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:23.957470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:24.000393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6c99a-80ce-4173-a946-690b7c084445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:24.003901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122b929-6b68-4047-8225-91e6eff7557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:24.005338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122b929-6b68-4047-8225-91e6eff7557a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.006441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122b929-6b68-4047-8225-91e6eff7557a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.007160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122b929-6b68-4047-8225-91e6eff7557a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.008733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122b929-6b68-4047-8225-91e6eff7557a None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=6de9ba88c5ff427caec1957e1305c2f6, user.id=7f091d543cf04f68b41e4456077bc426) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:24.009442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122b929-6b68-4047-8225-91e6eff7557a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:24.012165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122b929-6b68-4047-8225-91e6eff7557a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-60d6c99a-80ce-4173-a946-690b7c084445 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:24.013377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:24.016219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:24.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:24.026210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 89/182] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 668 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:24.028984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c588b-4595-4fd1-ac98-94a5471b5628 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.030531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c588b-4595-4fd1-ac98-94a5471b5628 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.030887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c588b-4595-4fd1-ac98-94a5471b5628 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.033364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 90/183] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:24.033882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 94/184] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:23 2022] GET /identity/v3/role_assignments?user.id=7f091d543cf04f68b41e4456077bc426&scope.project.id=6de9ba88c5ff427caec1957e1305c2f6&role.id=80a1f8f62542401a855036270bf2c51a => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:24.040295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ef52-d9b3-42d9-9c05-d873c24b6d7f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.040649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ef52-d9b3-42d9-9c05-d873c24b6d7f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.040828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ef52-d9b3-42d9-9c05-d873c24b6d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.044514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c588b-4595-4fd1-ac98-94a5471b5628 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:24.081232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1575ef52-d9b3-42d9-9c05-d873c24b6d7f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:24.082548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1575ef52-d9b3-42d9-9c05-d873c24b6d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:24.259636 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c588b-4595-4fd1-ac98-94a5471b5628 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:24.261287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 95/185] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:24.324155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:24.324792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.325065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.325341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.326027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:24.326332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:24.328485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:24.332583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:24.347556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 91/186] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:24.357558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575ef52-d9b3-42d9-9c05-d873c24b6d7f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:24.502147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575ef52-d9b3-42d9-9c05-d873c24b6d7f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:24.517816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:24.547336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af474973-370e-47f8-ad8f-bb3b6d9a536c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:24.548000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af474973-370e-47f8-ad8f-bb3b6d9a536c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.548280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af474973-370e-47f8-ad8f-bb3b6d9a536c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.548554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af474973-370e-47f8-ad8f-bb3b6d9a536c None admin] PATH_INFO: `/v3/users/3373df2ccc7a441a82be8841bd149de4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.549981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af474973-370e-47f8-ad8f-bb3b6d9a536c None admin] RBAC: Authorizing `identity:get_user(user_id=3373df2ccc7a441a82be8841bd149de4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:24.551637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af474973-370e-47f8-ad8f-bb3b6d9a536c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:24.553730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 96/187] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:24 2022] GET /identity/v3/users/3373df2ccc7a441a82be8841bd149de4 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:24.563659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:24.745098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93902f-373f-46c7-96e1-af4a5f5f7737 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:24.758661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224755f-a6ac-4b96-906a-bf5e88a49c15 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.758917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224755f-a6ac-4b96-906a-bf5e88a49c15 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.759219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224755f-a6ac-4b96-906a-bf5e88a49c15 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.760605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 97/188] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:24.767540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ad647-3381-43e6-af6e-72f01f66b60e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.768064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ad647-3381-43e6-af6e-72f01f66b60e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.768212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ad647-3381-43e6-af6e-72f01f66b60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.791467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-701ad647-3381-43e6-af6e-72f01f66b60e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:24.792698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-701ad647-3381-43e6-af6e-72f01f66b60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:24.843860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c3c49-bc04-4fbf-9bf6-f7185b9b4a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:24.844949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c3c49-bc04-4fbf-9bf6-f7185b9b4a7f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:24.845431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c3c49-bc04-4fbf-9bf6-f7185b9b4a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:24.845909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c3c49-bc04-4fbf-9bf6-f7185b9b4a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:24.890119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 98/189] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:24.890119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ad647-3381-43e6-af6e-72f01f66b60e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:24.984215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:25.000893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c3c49-bc04-4fbf-9bf6-f7185b9b4a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:25.004436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c3c49-bc04-4fbf-9bf6-f7185b9b4a7f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:25.131237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ad647-3381-43e6-af6e-72f01f66b60e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:25.155734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:25.211996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca9960-39f9-4257-a76a-630bf31133ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:25.212945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca9960-39f9-4257-a76a-630bf31133ee None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.213359 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca9960-39f9-4257-a76a-630bf31133ee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.221632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca9960-39f9-4257-a76a-630bf31133ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.221878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 92/190] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:25.221878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d262-9895-4453-92e0-775304232e81 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.221878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d262-9895-4453-92e0-775304232e81 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.221878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d262-9895-4453-92e0-775304232e81 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.223824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 93/191] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:25.224857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5d262-9895-4453-92e0-775304232e81 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:25.234584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca9960-39f9-4257-a76a-630bf31133ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f9ca9960-39f9-4257-a76a-630bf31133ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:25.244463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:25.246059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:25.247527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 99/192] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:25.250881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798138e-4c12-4936-ac5b-45e6640337a6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.250881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798138e-4c12-4936-ac5b-45e6640337a6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.250881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798138e-4c12-4936-ac5b-45e6640337a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.281313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1798138e-4c12-4936-ac5b-45e6640337a6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:25.282708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1798138e-4c12-4936-ac5b-45e6640337a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:25.386255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 100/193] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:25.387602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798138e-4c12-4936-ac5b-45e6640337a6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:25.397009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:25.406958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5d262-9895-4453-92e0-775304232e81 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:25.463494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:25.464341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.465743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.466181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.468373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:25.470126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:25.478367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 94/194] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:25 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:25.478367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:25.596552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:25.604042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798138e-4c12-4936-ac5b-45e6640337a6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:25.629279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf05d2-bbf9-4406-86ae-44dee60f79fd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:25.650833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:25.651457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.651739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.651992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.652906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:25.653622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:25.655499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:25.666332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 101/195] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:25.667244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:25.697768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:25.698467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.698776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.699085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.700222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:25.702377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:25.704563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 95/196] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:25.707439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:25.746251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:25.838491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ac3e-177e-4e88-9f6d-34a950e0ca95 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:25.839606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2d7d3-bbd9-48dc-8f10-45f69c4893e8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:25.884977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:25.885623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.885894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.886192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] PATH_INFO: `/v3/users/86bb2b3370da41bd96e171841b8aa033` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.887864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:25.888995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:25.889569 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:25.890097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:25.891547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] RBAC: Adding query filter params (role.id=64e702b1f13d4b62988575b426f8f8a3, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62, user.id=3373df2ccc7a441a82be8841bd149de4) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:25.892089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:25.894830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:25.905805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] RBAC: Authorizing `identity:get_user(user_id=86bb2b3370da41bd96e171841b8aa033)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:25.906159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 102/197] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:25 2022] GET /identity/v3/role_assignments?user.id=3373df2ccc7a441a82be8841bd149de4&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=64e702b1f13d4b62988575b426f8f8a3 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:25.906989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:25.907323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:25.909107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 96/198] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:25 2022] GET /identity/v3/users/86bb2b3370da41bd96e171841b8aa033 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:25.911634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:25.950381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:26.057306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:26.067406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf505-8064-4fcb-958b-aaec90f457d1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:26.073537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfaacb-191f-4991-8fe1-379636a06b22 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:26.115865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:26.116973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:26.117485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:26.118014 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:26.131777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:26.132439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:26.132731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:26.132967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:26.133782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:26.135328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:26.139229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:26.144353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:26.145468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:26.146571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 103/199] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:26.150212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:26.282744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 97/200] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:11:25 2022] POST /identity/v3/users => generated 297 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:26.284757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:26.301802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:26.325577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fa697-7427-4fea-9900-8fa03f6e3479 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:26.395604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:26.397535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:26.398023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:26.398490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:26.472381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fff1f0-be8f-4f9a-a8d6-cf16b2db318f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:26.516443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:26.517082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:26.517361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:26.518081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:26.518374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:26.518685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:26.520815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:26.530858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 98/201] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:26.532145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:26.534832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:26.539690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:26.562448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:26.668116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:26.694223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 104/202] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:26.703431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:26.722753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c5eca-7cc6-4d70-b56f-70c7c9be8907 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:26.783858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:26.785967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:26.786661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:26.787146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] PATH_INFO: `/v3/users/86bb2b3370da41bd96e171841b8aa033` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:26.789358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=86bb2b3370da41bd96e171841b8aa033)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:26.791642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:26.794338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 99/203] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:26 2022] GET /identity/v3/users/86bb2b3370da41bd96e171841b8aa033 => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:26.802126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:26.840129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163f931-974e-46d9-ab1c-6c68344caa8a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:26.880885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:26.881379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:26.881654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:26.881927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:26.883351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:26.884981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:26.887112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 105/204] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:26 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:26.899105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:26.906654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:26.938105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e3a93-1a06-4416-8121-73e2955e8ca5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:26.989285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:27.043108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:27.045064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:27.046830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:27.046995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:27.063922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7721c59-3001-4ae4-9850-e8af033b4419 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:27.112103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:27.112736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:27.113031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:27.113275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:27.114195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=86bb2b3370da41bd96e171841b8aa033) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:27.114513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:27.116037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:27.122724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 106/205] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:26 2022] GET /identity/v3/role_assignments?user.id=86bb2b3370da41bd96e171841b8aa033&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:27.151432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:27.153101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:27.252791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:27.284255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 100/206] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 14:11:27.290013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:27.445133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6889-8859-4b61-a8cc-f695df70d7dc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:27.480785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:27.520642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857ffb3-1695-464a-9f02-457a987bb7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:27.521692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857ffb3-1695-464a-9f02-457a987bb7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:27.522201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857ffb3-1695-464a-9f02-457a987bb7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:27.522661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857ffb3-1695-464a-9f02-457a987bb7f9 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:27.525127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857ffb3-1695-464a-9f02-457a987bb7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:27.527990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857ffb3-1695-464a-9f02-457a987bb7f9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:27.532032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 107/207] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:27 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:27.538734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:27.713090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40e860-e5d8-4e10-8573-c69e6f5c0853 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:27.781698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e60e70-c704-4d83-b806-ecad6dfe059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:27.782763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60e70-c704-4d83-b806-ecad6dfe059e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:27.783237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60e70-c704-4d83-b806-ecad6dfe059e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:27.783717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60e70-c704-4d83-b806-ecad6dfe059e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:27.785095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e60e70-c704-4d83-b806-ecad6dfe059e None admin] RBAC: Adding query filter params (role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=86bb2b3370da41bd96e171841b8aa033) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:27.785894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e60e70-c704-4d83-b806-ecad6dfe059e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:27.788655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e60e70-c704-4d83-b806-ecad6dfe059e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:27.800559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 101/208] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:27 2022] GET /identity/v3/role_assignments?user.id=86bb2b3370da41bd96e171841b8aa033&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:28.682890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645987-d223-485a-bc62-643bb29f5747 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:28.683432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645987-d223-485a-bc62-643bb29f5747 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:28.683721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645987-d223-485a-bc62-643bb29f5747 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:28.686300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 108/209] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:28.694741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5dad-5db3-4be3-981b-45f5c4c75899 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:28.695058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5dad-5db3-4be3-981b-45f5c4c75899 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:28.695291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5dad-5db3-4be3-981b-45f5c4c75899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:28.716268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4d0b5dad-5db3-4be3-981b-45f5c4c75899 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:28.718193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4d0b5dad-5db3-4be3-981b-45f5c4c75899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:28.759742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:28.797833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 102/210] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:29.145419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7645987-d223-485a-bc62-643bb29f5747 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:29.323994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7645987-d223-485a-bc62-643bb29f5747 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:29.396563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1863e-04a1-49d5-93a0-bd3cdb2a8e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:29.397576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1863e-04a1-49d5-93a0-bd3cdb2a8e56 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:29.398114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1863e-04a1-49d5-93a0-bd3cdb2a8e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:29.398606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1863e-04a1-49d5-93a0-bd3cdb2a8e56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:29.421317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1863e-04a1-49d5-93a0-bd3cdb2a8e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-60d1863e-04a1-49d5-93a0-bd3cdb2a8e56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:29.431336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:29.433149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:29.434833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 109/211] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:29.442083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b5dad-5db3-4be3-981b-45f5c4c75899 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:29.457394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce2cd8-df91-4563-86a5-fce50d369f12 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:29.457890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce2cd8-df91-4563-86a5-fce50d369f12 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:29.458445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce2cd8-df91-4563-86a5-fce50d369f12 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:29.461450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 110/212] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:29.468084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4a461-9a51-4dc7-9d53-b7a1989c2092 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:29.468639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4a461-9a51-4dc7-9d53-b7a1989c2092 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:29.469071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4a461-9a51-4dc7-9d53-b7a1989c2092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:29.507511 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-71b4a461-9a51-4dc7-9d53-b7a1989c2092 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:29.509938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-71b4a461-9a51-4dc7-9d53-b7a1989c2092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:29.603703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:29.624448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b5dad-5db3-4be3-981b-45f5c4c75899 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:29.665166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 111/213] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:29.691349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:29.692290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:29.692770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:29.693252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:29.694520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:29.694943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:29.697436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:29.713667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 103/214] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:29.721402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4a461-9a51-4dc7-9d53-b7a1989c2092 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:29.890298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4a461-9a51-4dc7-9d53-b7a1989c2092 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:29.933504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017051-f9a0-49a6-a16e-1e2ae89aa4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:29.934431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017051-f9a0-49a6-a16e-1e2ae89aa4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:29.934879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017051-f9a0-49a6-a16e-1e2ae89aa4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:29.935296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017051-f9a0-49a6-a16e-1e2ae89aa4f8 None admin] PATH_INFO: `/v3/users/58b95e32499d46ed9df1a12de7946fde` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:29.953476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26017051-f9a0-49a6-a16e-1e2ae89aa4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=58b95e32499d46ed9df1a12de7946fde)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:29.962047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26017051-f9a0-49a6-a16e-1e2ae89aa4f8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:29.962047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 112/215] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:29 2022] GET /identity/v3/users/58b95e32499d46ed9df1a12de7946fde => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:29.970504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:29.985299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b790dac-eeec-4f45-a5fa-45040cb6a099 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:29.985796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b790dac-eeec-4f45-a5fa-45040cb6a099 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:29.986343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b790dac-eeec-4f45-a5fa-45040cb6a099 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:29.988680 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 113/216] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:29.997828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825678-b98a-479a-a7b4-81d02e3d271c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:29.998890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825678-b98a-479a-a7b4-81d02e3d271c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:30.000267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825678-b98a-479a-a7b4-81d02e3d271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:30.016615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:30.031068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-39825678-b98a-479a-a7b4-81d02e3d271c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:30.032830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-39825678-b98a-479a-a7b4-81d02e3d271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:30.066987 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:30.103738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b21841-2f0c-4c80-aaf4-4625f2ed00c2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:30.129737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 114/217] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:30.131298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825678-b98a-479a-a7b4-81d02e3d271c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:30.144227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff3d88-80a9-463d-bd40-fd9540f7b2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:30.145101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff3d88-80a9-463d-bd40-fd9540f7b2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:30.145517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff3d88-80a9-463d-bd40-fd9540f7b2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:30.145950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff3d88-80a9-463d-bd40-fd9540f7b2a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:30.265138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff3d88-80a9-463d-bd40-fd9540f7b2a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:30.268242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff3d88-80a9-463d-bd40-fd9540f7b2a2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:30.296302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825678-b98a-479a-a7b4-81d02e3d271c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:30.334813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:30.340714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a224f-5691-4023-9c9c-e7684e964a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:30.348426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a224f-5691-4023-9c9c-e7684e964a9a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:30.348426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a224f-5691-4023-9c9c-e7684e964a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:30.348426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a224f-5691-4023-9c9c-e7684e964a9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:30.348426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a224f-5691-4023-9c9c-e7684e964a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:30.348426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a224f-5691-4023-9c9c-e7684e964a9a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:30.348426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 115/218] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:30.352359 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648a048-b95b-4250-b8a9-632df96a31d2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:30.352803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648a048-b95b-4250-b8a9-632df96a31d2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:30.353281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648a048-b95b-4250-b8a9-632df96a31d2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:30.355864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 116/219] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:30.356910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648a048-b95b-4250-b8a9-632df96a31d2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:30.410729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:30.460817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 104/220] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:30.460817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e08c3d-2c68-4187-b340-b6466ae70267 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:30.460817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e08c3d-2c68-4187-b340-b6466ae70267 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:30.460817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e08c3d-2c68-4187-b340-b6466ae70267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:30.479321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b9e08c3d-2c68-4187-b340-b6466ae70267 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:30.480675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b9e08c3d-2c68-4187-b340-b6466ae70267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:30.521482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648a048-b95b-4250-b8a9-632df96a31d2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:30.559684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 105/221] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:30.561096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e08c3d-2c68-4187-b340-b6466ae70267 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:30.577947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:30.579546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:30.580226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:30.580894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:30.582035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:30.585332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:30.643034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:30.701792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e08c3d-2c68-4187-b340-b6466ae70267 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:30.714063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 117/222] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:11:30 2022] POST /identity/v3/users => generated 297 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:30.714063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:30.782594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:30.785324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:30.786546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:30.787413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:30.816045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:30.827599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:30.829229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:30.830720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:30.835179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 106/223] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:30.842405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:30.925332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:30.944953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049223eb-31db-4e53-b79e-c8e737045f47 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:31.002044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153b07d-e6f1-4c01-ac26-43ea79e7e045 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:31.006143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:31.006143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:31.006143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:31.006143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:31.006543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:31.009086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:31.012213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 118/224] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:30 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:31.014670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:31.035011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:31.053447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:31.054146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:31.054433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:31.054708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:31.055539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:31.055833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:31.058050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:31.067743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 107/225] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:31.068624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:31.191231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e7de2-911a-4fc3-8497-fd382a913e54 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:31.220006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e9239-5f6d-4ece-8457-d66231e38690 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:31.249609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:31.250214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:31.250500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:31.250771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:31.264514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:31.270074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:31.271929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 119/226] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:31.273443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:31.296322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:31.298632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:31.299782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:31.300468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:31.303502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:31.304905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=58b95e32499d46ed9df1a12de7946fde) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:31.306183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:31.309861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:31.325457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 108/227] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:31 2022] GET /identity/v3/role_assignments?user.id=58b95e32499d46ed9df1a12de7946fde&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=80a1f8f62542401a855036270bf2c51a => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:31.331799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:31.539283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:31.593768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d18f3-e7e2-401d-a7c4-007c7bb0ddd0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:31.663948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377185db-8b57-4202-8c04-b09d2d5e0e23 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:31.671730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:31.673318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:31.674122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:31.674886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] PATH_INFO: `/v3/users/86bb2b3370da41bd96e171841b8aa033` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:31.677379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] RBAC: Authorizing `identity:get_user(user_id=86bb2b3370da41bd96e171841b8aa033)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:31.680637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:31.684603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 120/228] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:31 2022] GET /identity/v3/users/86bb2b3370da41bd96e171841b8aa033 => generated 283 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:31.695222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:31.716542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:31.718386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:31.718386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:31.718718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:31.719275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:31.719583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:31.721077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:31.730473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 109/229] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:31.737446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:31.860575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:31.884527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f42975-26dd-4724-96a6-c93c2404f5cd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:31.943714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:31.944538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:31.944799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:31.944924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:31.946190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:31.947679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:31.949381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 121/230] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:31 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:31.965472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:32.052356 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3707b4-3542-4e21-b919-fe3beb2d42fc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:32.082386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:32.142272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:32.145151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:32.145429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:32.145703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] PATH_INFO: `/v3/users/86bb2b3370da41bd96e171841b8aa033` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:32.156412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] RBAC: Authorizing `identity:get_user(user_id=86bb2b3370da41bd96e171841b8aa033)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:32.158051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:32.160138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 110/231] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:31 2022] GET /identity/v3/users/86bb2b3370da41bd96e171841b8aa033 => generated 283 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:32.165857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:32.230658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13635b-3cc8-42b6-8267-24ac14c0deda None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:32.361273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b540-16be-4a20-9ac8-ce662f0fa040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:32.361273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b540-16be-4a20-9ac8-ce662f0fa040 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:32.361273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b540-16be-4a20-9ac8-ce662f0fa040 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:32.361273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b540-16be-4a20-9ac8-ce662f0fa040 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/86bb2b3370da41bd96e171841b8aa033/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:32.368017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe63211-e6a6-4b77-9d31-af4c6e90cb0a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:32.375099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912b540-16be-4a20-9ac8-ce662f0fa040 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13307a6663b4948b0439bb4ebc46e3e, user_id=86bb2b3370da41bd96e171841b8aa033, role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:32.376615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912b540-16be-4a20-9ac8-ce662f0fa040 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:32.448378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:32.449575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:32.450148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:32.450657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:32.454125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:32.457399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:32.461505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 111/232] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:32 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:32.470511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:32.512519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 122/233] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:31 2022] PUT /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/86bb2b3370da41bd96e171841b8aa033/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:32.735505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:32.922215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b341c-ed90-442f-856d-c1a5e0aebcad None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:33.006914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3045b-652b-40dd-8e73-a5d507323a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:33.007838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3045b-652b-40dd-8e73-a5d507323a88 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:33.008291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3045b-652b-40dd-8e73-a5d507323a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:33.008746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3045b-652b-40dd-8e73-a5d507323a88 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/86bb2b3370da41bd96e171841b8aa033/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:33.012322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3045b-652b-40dd-8e73-a5d507323a88 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13307a6663b4948b0439bb4ebc46e3e, user_id=86bb2b3370da41bd96e171841b8aa033, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:33.015740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3045b-652b-40dd-8e73-a5d507323a88 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:33.140094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 112/234] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:32 2022] PUT /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/86bb2b3370da41bd96e171841b8aa033/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:33.320029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf3756-8a95-4d6a-b477-fc9726067e32 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:33.320528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf3756-8a95-4d6a-b477-fc9726067e32 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:33.320528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf3756-8a95-4d6a-b477-fc9726067e32 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:33.321895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 123/235] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:33.330295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee51f02-1f20-4945-82da-cc2989ebe926 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:33.330798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee51f02-1f20-4945-82da-cc2989ebe926 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:33.331302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee51f02-1f20-4945-82da-cc2989ebe926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:33.371414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2ee51f02-1f20-4945-82da-cc2989ebe926 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:33.372660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2ee51f02-1f20-4945-82da-cc2989ebe926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:33.436191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab86b55-e5f2-49e2-81e6-6b30105ae992 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:33.436607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab86b55-e5f2-49e2-81e6-6b30105ae992 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:33.436986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab86b55-e5f2-49e2-81e6-6b30105ae992 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:33.438961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 124/236] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:33.445004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b00e15-e669-4aa2-817d-204b868922fa None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:33.445296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b00e15-e669-4aa2-817d-204b868922fa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:33.445558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b00e15-e669-4aa2-817d-204b868922fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:33.471822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-78b00e15-e669-4aa2-817d-204b868922fa None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:33.473896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-78b00e15-e669-4aa2-817d-204b868922fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:33.477764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:33.479324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 113/237] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:33.596395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 125/238] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:33.955681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee51f02-1f20-4945-82da-cc2989ebe926 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:33.996880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:34.011574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b00e15-e669-4aa2-817d-204b868922fa None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:34.156350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee51f02-1f20-4945-82da-cc2989ebe926 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:34.231073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01245984-fdac-4362-b774-892521714441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:34.231650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01245984-fdac-4362-b774-892521714441 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:34.231919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01245984-fdac-4362-b774-892521714441 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:34.232180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01245984-fdac-4362-b774-892521714441 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:34.234559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01245984-fdac-4362-b774-892521714441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:34.236165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01245984-fdac-4362-b774-892521714441 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:34.238038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 114/239] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:34.244862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01245984-fdac-4362-b774-892521714441 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:34.266920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b00e15-e669-4aa2-817d-204b868922fa None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:34.334911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4df45-5104-4aee-840e-9b30232efc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:34.335881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4df45-5104-4aee-840e-9b30232efc74 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:34.335881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4df45-5104-4aee-840e-9b30232efc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:34.336183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4df45-5104-4aee-840e-9b30232efc74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:34.348729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:34.349834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4df45-5104-4aee-840e-9b30232efc74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-82b4df45-5104-4aee-840e-9b30232efc74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:34.354824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:34.356524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 126/240] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:34.358255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36f06-f0c2-4c56-8165-42374c85a40c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:34.368608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36f06-f0c2-4c56-8165-42374c85a40c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:34.368608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36f06-f0c2-4c56-8165-42374c85a40c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:34.368608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 127/241] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:34.368608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b36f06-f0c2-4c56-8165-42374c85a40c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:34.472640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01245984-fdac-4362-b774-892521714441 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:34.475813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:34.515874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:34.516407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:34.516679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:34.516994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:34.517651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:34.519267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:34.523504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b36f06-f0c2-4c56-8165-42374c85a40c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:34.590043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d52921-2e8d-438b-a980-7b3e51aa252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:34.591082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d52921-2e8d-438b-a980-7b3e51aa252c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:34.591573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d52921-2e8d-438b-a980-7b3e51aa252c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:34.592607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d52921-2e8d-438b-a980-7b3e51aa252c None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:34.593902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d52921-2e8d-438b-a980-7b3e51aa252c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:34.594458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d52921-2e8d-438b-a980-7b3e51aa252c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:34.597089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d52921-2e8d-438b-a980-7b3e51aa252c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:34.613688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 128/242] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:34.620666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6cd48-5989-4caf-bb0d-884aaee1fb00 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:34.620666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6cd48-5989-4caf-bb0d-884aaee1fb00 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:34.620666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6cd48-5989-4caf-bb0d-884aaee1fb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:34.639911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 115/243] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:11:34 2022] POST /identity/v3/users => generated 293 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:34.641261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:34.644177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3de6cd48-5989-4caf-bb0d-884aaee1fb00 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:34.646499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3de6cd48-5989-4caf-bb0d-884aaee1fb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:34.785484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:34.807952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 129/244] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:34.847675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f28d82-bf5a-4721-b07a-5e43a708a9eb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:34.919251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:34.921100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:34.921576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:34.922084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] PATH_INFO: `/v3/users/fdbb6920786d4f3a9318537b1decb75e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:34.950423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] RBAC: Authorizing `identity:get_user(user_id=fdbb6920786d4f3a9318537b1decb75e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:34.953980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:34.957235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 116/245] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:34 2022] GET /identity/v3/users/fdbb6920786d4f3a9318537b1decb75e => generated 297 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:34.963213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6cd48-5989-4caf-bb0d-884aaee1fb00 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:35.120460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6cd48-5989-4caf-bb0d-884aaee1fb00 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:35.174401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:35.175267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:35.175685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:35.176095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:35.221451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:35.247682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:35.270897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:35.284998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:35.287253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:35.348366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3821e-3d89-49cd-8167-10e98abab2d7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:35.400645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:35.401501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:35.402066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:35.402419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:35.422676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:35.432463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:35.433600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:35.434520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 117/246] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:35.442084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:35.512518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 130/247] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:35.520690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:35.597957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:35.631893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48880d1e-36c0-4e91-8e8e-fd36f4b0843e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:35.654162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:35.666706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c76748-8bf2-4ade-b026-8a0da28ad7ac None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:35.695794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd984a39-26b6-4057-8a34-149d813cb8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:35.696438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd984a39-26b6-4057-8a34-149d813cb8ff None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:35.696716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd984a39-26b6-4057-8a34-149d813cb8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:35.696989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd984a39-26b6-4057-8a34-149d813cb8ff None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:35.697681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd984a39-26b6-4057-8a34-149d813cb8ff None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:35.698009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd984a39-26b6-4057-8a34-149d813cb8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:35.699445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd984a39-26b6-4057-8a34-149d813cb8ff None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:35.721456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 118/248] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:35.722622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf862d9-4015-4faf-8a06-cc7390e5ba9c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:35.722909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf862d9-4015-4faf-8a06-cc7390e5ba9c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:35.723180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf862d9-4015-4faf-8a06-cc7390e5ba9c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:35.730482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 119/249] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:35.731658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf862d9-4015-4faf-8a06-cc7390e5ba9c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:35.752401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448087cf-391e-421a-9320-0195a97a7900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:35.753097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448087cf-391e-421a-9320-0195a97a7900 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:35.753382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448087cf-391e-421a-9320-0195a97a7900 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:35.753655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448087cf-391e-421a-9320-0195a97a7900 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:35.754892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448087cf-391e-421a-9320-0195a97a7900 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:35.756425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448087cf-391e-421a-9320-0195a97a7900 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:35.762445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 131/250] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:35 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:35.765697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8f532-ef7f-4402-8ecc-9158a29d5dcc None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:35.766323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8f532-ef7f-4402-8ecc-9158a29d5dcc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:35.768604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8f532-ef7f-4402-8ecc-9158a29d5dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:35.801980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2fb8f532-ef7f-4402-8ecc-9158a29d5dcc None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:35.804102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2fb8f532-ef7f-4402-8ecc-9158a29d5dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:35.911060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 132/251] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:35.912491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8f532-ef7f-4402-8ecc-9158a29d5dcc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:35.946656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf862d9-4015-4faf-8a06-cc7390e5ba9c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:36.031492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:36.032534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:36.033036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:36.033521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] PATH_INFO: `/v3/users/58b95e32499d46ed9df1a12de7946fde` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:36.035047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] RBAC: Authorizing `identity:get_user(user_id=58b95e32499d46ed9df1a12de7946fde)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:36.037358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:36.039753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 120/252] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:35 2022] GET /identity/v3/users/58b95e32499d46ed9df1a12de7946fde => generated 297 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:36.057494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:36.076057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:36.106925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:36.124958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8f532-ef7f-4402-8ecc-9158a29d5dcc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:36.194101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:36.194101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:36.194101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:36.194101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:36.194477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] RBAC: Adding query filter params (role.id=64e702b1f13d4b62988575b426f8f8a3, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=fdbb6920786d4f3a9318537b1decb75e) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:36.194477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:36.195940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:36.207064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 133/253] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:35 2022] GET /identity/v3/role_assignments?user.id=fdbb6920786d4f3a9318537b1decb75e&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=64e702b1f13d4b62988575b426f8f8a3 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:36.312725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:36.370775 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9e5b5-85c4-457e-ab18-59152e0993d9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:36.493496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a808b18-8c3e-40de-8b20-6e6be8811a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:36.494642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808b18-8c3e-40de-8b20-6e6be8811a1f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:36.495121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808b18-8c3e-40de-8b20-6e6be8811a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:36.495606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808b18-8c3e-40de-8b20-6e6be8811a1f None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:36.498876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a808b18-8c3e-40de-8b20-6e6be8811a1f None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:36.502346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a808b18-8c3e-40de-8b20-6e6be8811a1f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:36.506670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 121/254] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:36 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:36.509309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8481b4d-c0dd-4221-be4f-e91783073509 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:36.509800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8481b4d-c0dd-4221-be4f-e91783073509 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:36.510322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8481b4d-c0dd-4221-be4f-e91783073509 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:36.514454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 122/255] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:36.522023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8481b4d-c0dd-4221-be4f-e91783073509 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:36.657169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294c224-d39a-4bfc-a762-f60062158ce6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:36.710083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:36.752593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e196e-f3e5-4bc2-8b74-55cde9facf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:36.753156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e196e-f3e5-4bc2-8b74-55cde9facf5d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:36.753432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e196e-f3e5-4bc2-8b74-55cde9facf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:36.753700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e196e-f3e5-4bc2-8b74-55cde9facf5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:36.766503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8481b4d-c0dd-4221-be4f-e91783073509 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:36.800676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e196e-f3e5-4bc2-8b74-55cde9facf5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ae8e196e-f3e5-4bc2-8b74-55cde9facf5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:36.822764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:36.823759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:36.824547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:36.825156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 134/256] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 513 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:36.827257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8978-3281-4451-af5c-87e013ce3998 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:36.828521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8978-3281-4451-af5c-87e013ce3998 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:36.828990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8978-3281-4451-af5c-87e013ce3998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:36.865718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b52e8978-3281-4451-af5c-87e013ce3998 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:36.867835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b52e8978-3281-4451-af5c-87e013ce3998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:36.887993 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:36.894811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:36.895514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:36.895799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:36.898896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/58b95e32499d46ed9df1a12de7946fde/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:36.902926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13307a6663b4948b0439bb4ebc46e3e, user_id=58b95e32499d46ed9df1a12de7946fde, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:36.904972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:36.998360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 135/257] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:37.000043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e8978-3281-4451-af5c-87e013ce3998 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:37.000927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 123/258] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:36 2022] PUT /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/58b95e32499d46ed9df1a12de7946fde/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:37.148706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e8978-3281-4451-af5c-87e013ce3998 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:37.192067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:37.193049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:37.193542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:37.194039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:37.195379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:37.195939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:37.198620 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:37.215380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 136/259] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:37.224707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:37.415084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdfd3c-add3-43a1-8060-45589e076018 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:37.463317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:37.463977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:37.464255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:37.464526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] PATH_INFO: `/v3/users/86bb2b3370da41bd96e171841b8aa033` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:37.470953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] RBAC: Authorizing `identity:get_user(user_id=86bb2b3370da41bd96e171841b8aa033)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:37.472446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:37.474681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 124/260] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:37 2022] GET /identity/v3/users/86bb2b3370da41bd96e171841b8aa033 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:37.484262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:37.518029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:37.747262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ff48-9eea-4615-b489-784c27466374 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:37.814192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:37.814833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:37.815147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:37.815447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:37.900821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:37.902368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:37.911791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:37.954724 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:37.958754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:38.020793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 137/261] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:38.026709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:38.144461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec0607-ab9c-47c5-9a83-c3b3f707cf01 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:38.146374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70be17-1992-43e2-9a4d-bdae0263427e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:38.153056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:38.189262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fe9ea-b15e-473e-ad03-39c72dd23b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:38.191293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe9ea-b15e-473e-ad03-39c72dd23b2f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.191778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe9ea-b15e-473e-ad03-39c72dd23b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.192235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe9ea-b15e-473e-ad03-39c72dd23b2f None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.193505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fe9ea-b15e-473e-ad03-39c72dd23b2f None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:38.195259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fe9ea-b15e-473e-ad03-39c72dd23b2f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:38.196230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec9f69-f870-4d55-8a41-6450f8204cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:38.196895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec9f69-f870-4d55-8a41-6450f8204cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.197171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec9f69-f870-4d55-8a41-6450f8204cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.197442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec9f69-f870-4d55-8a41-6450f8204cc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.202152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 138/262] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:38 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:38.204022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c39fe0-eccc-484a-b849-26c6b647e1dc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.204488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c39fe0-eccc-484a-b849-26c6b647e1dc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.204919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c39fe0-eccc-484a-b849-26c6b647e1dc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.206622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 139/263] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:38.209956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c39fe0-eccc-484a-b849-26c6b647e1dc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:38.211595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec9f69-f870-4d55-8a41-6450f8204cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-adec9f69-f870-4d55-8a41-6450f8204cc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:38.218726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:38.219755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:38.220814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 125/264] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:38.221743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b24f0-22bb-4af8-bc4e-5b995ad719b6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.222046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b24f0-22bb-4af8-bc4e-5b995ad719b6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.222304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b24f0-22bb-4af8-bc4e-5b995ad719b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.245663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-304b24f0-22bb-4af8-bc4e-5b995ad719b6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:38.246969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-304b24f0-22bb-4af8-bc4e-5b995ad719b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:38.353583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:38.354032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c39fe0-eccc-484a-b849-26c6b647e1dc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:38.357214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 126/265] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:38.361377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b24f0-22bb-4af8-bc4e-5b995ad719b6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:38.465832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:38.467029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.467536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.468020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.468962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] RBAC: Adding query filter params (role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=86bb2b3370da41bd96e171841b8aa033) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:38.469497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:38.471268 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:38.485759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 140/266] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:38 2022] GET /identity/v3/role_assignments?user.id=86bb2b3370da41bd96e171841b8aa033&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:38.606751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b24f0-22bb-4af8-bc4e-5b995ad719b6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:38.666727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:38.667384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.667666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.667938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.668651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:38.668956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:38.671581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:38.688150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 127/267] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:38.693822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:38.777093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:38.778398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:38.863061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:38.895095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bead1b-dfea-4a2a-8f0b-fe0d01cdf9ef None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:38.922014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce6a8e-b9be-4771-8da0-9d40a91db1f7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:38.948107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b6071-c9c7-42fe-8ad2-9baf9d448ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:38.949114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b6071-c9c7-42fe-8ad2-9baf9d448ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.949558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b6071-c9c7-42fe-8ad2-9baf9d448ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.950028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b6071-c9c7-42fe-8ad2-9baf9d448ae8 None admin] PATH_INFO: `/v3/users/86bb2b3370da41bd96e171841b8aa033` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.952422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b6071-c9c7-42fe-8ad2-9baf9d448ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=86bb2b3370da41bd96e171841b8aa033)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:38.954328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b6071-c9c7-42fe-8ad2-9baf9d448ae8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:38.956758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 141/268] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:38 2022] GET /identity/v3/users/86bb2b3370da41bd96e171841b8aa033 => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:38.958321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ff2e-a556-4ffc-8141-b1639e6c8764 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.958825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ff2e-a556-4ffc-8141-b1639e6c8764 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.959300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3ff2e-a556-4ffc-8141-b1639e6c8764 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.965715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 142/269] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:38.966133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0559e64-6259-4b42-a5a9-5df7a56f208d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:38.967527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559e64-6259-4b42-a5a9-5df7a56f208d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.967895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559e64-6259-4b42-a5a9-5df7a56f208d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.968255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559e64-6259-4b42-a5a9-5df7a56f208d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:38.968835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ff2e-a556-4ffc-8141-b1639e6c8764 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:38.981540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0559e64-6259-4b42-a5a9-5df7a56f208d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a0559e64-6259-4b42-a5a9-5df7a56f208d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:38.988771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:38.989744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:38.990585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 128/270] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:38.991909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ea43-74e7-4e2c-9b29-5bedf20e4a72 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:38.992326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ea43-74e7-4e2c-9b29-5bedf20e4a72 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:38.992663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ea43-74e7-4e2c-9b29-5bedf20e4a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:39.013558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fc20ea43-74e7-4e2c-9b29-5bedf20e4a72 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:39.015970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fc20ea43-74e7-4e2c-9b29-5bedf20e4a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:39.101084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 129/271] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:39.102112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20ea43-74e7-4e2c-9b29-5bedf20e4a72 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:39.102794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3ff2e-a556-4ffc-8141-b1639e6c8764 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:39.113345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:39.150123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:39.151216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:39.151764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:39.152208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:39.173891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:39.310598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:39.313104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:39.315039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20ea43-74e7-4e2c-9b29-5bedf20e4a72 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:39.443445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 143/272] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:39.450226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:39.461482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:39.471516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:39.472391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:39.475215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:39.475215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:39.475215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:39.475215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:39.476205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:39.489337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 130/273] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:39.490667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:39.593683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e526b7-3c01-40a2-bdb4-7d41359be63b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:39.637297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014b8f4-07ee-4b18-8c51-1ac99f51ae9f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:39.639732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:39.640665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:39.641189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:39.641687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:39.643274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:39.644977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:39.646979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 144/274] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:39 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:39.652041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:39.653433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:39.689980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:39.691039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:39.691524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:39.692022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:39.693617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:39.695371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:39.697281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 131/275] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:39.698361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:39.767514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:39.806209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15eb4e-2563-4924-a43c-f59530a5b49c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:39.833268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb9bbf-6310-458e-84fa-8ea6784acfbf None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:39.853001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:39.854120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:39.854612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:39.855127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] PATH_INFO: `/v3/users/5fcb0eae27e74ed295b197f3f34c06d8` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:39.874098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] RBAC: Authorizing `identity:get_user(user_id=5fcb0eae27e74ed295b197f3f34c06d8)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:39.876506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:39.877013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:39.877138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:39.877415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:39.877690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:39.878468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=86bb2b3370da41bd96e171841b8aa033) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:39.878784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:39.880126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 145/276] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:39 2022] GET /identity/v3/users/5fcb0eae27e74ed295b197f3f34c06d8 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:39.880670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:39.881497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:39.888310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 132/277] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:39 2022] GET /identity/v3/role_assignments?user.id=86bb2b3370da41bd96e171841b8aa033&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:39.894267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:39.951482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:40.039515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7aec66-3d18-463b-aae0-d8a8df844a28 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:40.047679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d675c2-0dc6-4474-a7f0-7ea31a39ed08 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:40.095076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edf0b6-988b-421e-9e18-f978661e59f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:40.097124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:40.097488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf0b6-988b-421e-9e18-f978661e59f8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.098615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf0b6-988b-421e-9e18-f978661e59f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.098751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf0b6-988b-421e-9e18-f978661e59f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.099492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edf0b6-988b-421e-9e18-f978661e59f8 None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, user.id=5fcb0eae27e74ed295b197f3f34c06d8, scope.system=all) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:40.099806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edf0b6-988b-421e-9e18-f978661e59f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:40.101327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edf0b6-988b-421e-9e18-f978661e59f8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:40.108625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe98fe7-e246-4bfa-b082-c4bf623a3bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:40.113825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 133/278] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 24 14:11:39 2022] GET /identity/v3/role_assignments?user.id=5fcb0eae27e74ed295b197f3f34c06d8&scope.system=all&role.id=80a1f8f62542401a855036270bf2c51a => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:40.113978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe98fe7-e246-4bfa-b082-c4bf623a3bc8 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.113978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe98fe7-e246-4bfa-b082-c4bf623a3bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.113978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe98fe7-e246-4bfa-b082-c4bf623a3bc8 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.114225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c961-63ac-4064-beac-323cb9467b99 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.114225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c961-63ac-4064-beac-323cb9467b99 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.114225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c961-63ac-4064-beac-323cb9467b99 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.114406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe98fe7-e246-4bfa-b082-c4bf623a3bc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:40.114473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 134/279] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:40.114538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe98fe7-e246-4bfa-b082-c4bf623a3bc8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:40.118944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2548f04-abfa-49e6-9cc2-08db0082fb1c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.119301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2548f04-abfa-49e6-9cc2-08db0082fb1c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.119503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2548f04-abfa-49e6-9cc2-08db0082fb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.144440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f2548f04-abfa-49e6-9cc2-08db0082fb1c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:40.145035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f2548f04-abfa-49e6-9cc2-08db0082fb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:40.244641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 146/280] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:11:39 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:40.250136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7b8c-3274-4db4-87fd-3e92ea4aa204 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.251238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7b8c-3274-4db4-87fd-3e92ea4aa204 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.252324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7b8c-3274-4db4-87fd-3e92ea4aa204 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.257729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 147/281] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:40.267306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3acaa5-ff5c-4501-9001-781b60b6d0c2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.267763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3acaa5-ff5c-4501-9001-781b60b6d0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.268228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3acaa5-ff5c-4501-9001-781b60b6d0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.270518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 135/282] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:40.299210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-dc3acaa5-ff5c-4501-9001-781b60b6d0c2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:40.301248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dc3acaa5-ff5c-4501-9001-781b60b6d0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:40.421333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 148/283] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:40.570980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2548f04-abfa-49e6-9cc2-08db0082fb1c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:40.613036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:40.702635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2548f04-abfa-49e6-9cc2-08db0082fb1c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:40.755224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:40.755891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.756172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.756442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.773216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:40.781229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:40.784099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:40.790484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 136/284] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:40.792776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3acaa5-ff5c-4501-9001-781b60b6d0c2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:40.914880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3acaa5-ff5c-4501-9001-781b60b6d0c2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:40.939613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:40.962255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:40.962795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:40.963055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:40.963334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:40.964028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:40.964334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:40.966015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:40.971086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:40.979445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 149/285] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:40.987162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:41.121217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8eb08-5cf6-4d8a-901c-3857fd830474 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:41.168410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:41.169091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:41.169381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:41.170440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] PATH_INFO: `/v3/users/58b95e32499d46ed9df1a12de7946fde` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:41.172848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] RBAC: Authorizing `identity:get_user(user_id=58b95e32499d46ed9df1a12de7946fde)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:41.174758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:41.176941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 150/286] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:40 2022] GET /identity/v3/users/58b95e32499d46ed9df1a12de7946fde => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:41.184150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:41.287317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:41.302391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:41.317295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16a8c-e182-4c6e-842c-a0f9b2cd962c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:41.330245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9fd64-8a7e-4b6c-9df1-8a49b19d459e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:41.392249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:41.392889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:41.393172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:41.393445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:41.416662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:41.417738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:41.418260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:41.418742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:41.447203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:41.453720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:41.454889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:41.455654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:41.458778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 137/287] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 488 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:41.467301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:41.542444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:41.546024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:41.662560 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:41.694942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1d6b4-558f-4f63-b5ce-09bd1489403f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:41.771824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 151/288] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:41.774788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:41.775551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:41.775684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:41.776029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:41.780164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:41.780164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:41.780501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:41.780666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:41.798232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 138/289] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:41.803613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:41.842798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:42.020920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7926eb-1236-4595-872c-7177adb56df2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:42.109218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:42.110819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:42.111598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:42.112365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:42.115905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:42.119600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:42.124548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 152/290] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:41 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:42.136353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:42.159561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fbb0b-4c53-4056-a308-7a4d4d615929 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:42.190193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:42.286879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:42.288451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:42.291264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:42.293747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] PATH_INFO: `/v3/users/fdbb6920786d4f3a9318537b1decb75e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:42.299530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=fdbb6920786d4f3a9318537b1decb75e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:42.302629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:42.306403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 139/291] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:41 2022] GET /identity/v3/users/fdbb6920786d4f3a9318537b1decb75e => generated 297 bytes in 503 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:42.314510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:42.443424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d81bf-bed7-484a-88e9-f3e6401d5b5a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:42.559806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:42.562185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:42.563182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:42.563945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:42.566018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=58b95e32499d46ed9df1a12de7946fde) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:42.566875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:42.569926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:42.587105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 153/292] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:42 2022] GET /identity/v3/role_assignments?user.id=58b95e32499d46ed9df1a12de7946fde&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=80a1f8f62542401a855036270bf2c51a => generated 608 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:42.665978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:42.709672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a51050-6bc2-4a6c-a5b1-7773da38a3b2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:42.809381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffe655-a4f5-4b68-a0ee-005115b728b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:42.811032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe655-a4f5-4b68-a0ee-005115b728b6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:42.811772 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe655-a4f5-4b68-a0ee-005115b728b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:42.812255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe655-a4f5-4b68-a0ee-005115b728b6 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:42.814605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffe655-a4f5-4b68-a0ee-005115b728b6 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:42.817520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffe655-a4f5-4b68-a0ee-005115b728b6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:42.821252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 140/293] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:42 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:42.827364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:43.006659 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:43.020775 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28c324-5802-4364-8483-82aae559297f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.021259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28c324-5802-4364-8483-82aae559297f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.021746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28c324-5802-4364-8483-82aae559297f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.025764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 141/294] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:43.037320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb576-abfb-4eb0-8267-d5e3a6a32dec None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:43.038463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7173b-a70d-4ef7-8b79-595ed7b86aac None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.038966 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7173b-a70d-4ef7-8b79-595ed7b86aac None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.039466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7173b-a70d-4ef7-8b79-595ed7b86aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.083867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7bf7173b-a70d-4ef7-8b79-595ed7b86aac None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:43.084970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7bf7173b-a70d-4ef7-8b79-595ed7b86aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:43.101660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a85fbc-6853-430e-9d4c-8e8b7465b51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:43.102409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85fbc-6853-430e-9d4c-8e8b7465b51f None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.102689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85fbc-6853-430e-9d4c-8e8b7465b51f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.102965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85fbc-6853-430e-9d4c-8e8b7465b51f None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/fdbb6920786d4f3a9318537b1decb75e/roles/64e702b1f13d4b62988575b426f8f8a3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.105143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a85fbc-6853-430e-9d4c-8e8b7465b51f None admin] RBAC: Authorizing `identity:create_grant(project_id=d13307a6663b4948b0439bb4ebc46e3e, user_id=fdbb6920786d4f3a9318537b1decb75e, role_id=64e702b1f13d4b62988575b426f8f8a3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:43.106702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a85fbc-6853-430e-9d4c-8e8b7465b51f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:43.185986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 142/295] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:43.187351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf55a7-492b-4142-9bc2-a402b8c42e88 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.187644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf55a7-492b-4142-9bc2-a402b8c42e88 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.187924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf55a7-492b-4142-9bc2-a402b8c42e88 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.189635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 143/296] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:43.198733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d11f2e-7968-4a2a-991b-aa6d271dc155 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.199078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d11f2e-7968-4a2a-991b-aa6d271dc155 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.201683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d11f2e-7968-4a2a-991b-aa6d271dc155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.204678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 154/297] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:11:42 2022] PUT /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/users/fdbb6920786d4f3a9318537b1decb75e/roles/64e702b1f13d4b62988575b426f8f8a3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:43.223194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a9d11f2e-7968-4a2a-991b-aa6d271dc155 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:43.224988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a9d11f2e-7968-4a2a-991b-aa6d271dc155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:43.391112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 144/298] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:43.494941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d26648-b27f-44d4-bc90-d593a7434c10 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.495247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d26648-b27f-44d4-bc90-d593a7434c10 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.495534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d26648-b27f-44d4-bc90-d593a7434c10 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.497246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 155/299] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:43.504102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4cb8-250e-4dec-8eef-a2f72cb85150 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.504408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4cb8-250e-4dec-8eef-a2f72cb85150 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.504659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4cb8-250e-4dec-8eef-a2f72cb85150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.525027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9a9c4cb8-250e-4dec-8eef-a2f72cb85150 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:43.526303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9a9c4cb8-250e-4dec-8eef-a2f72cb85150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:43.536797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:43.613952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 145/300] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:43.677455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d26648-b27f-44d4-bc90-d593a7434c10 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:43.711093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4cb8-250e-4dec-8eef-a2f72cb85150 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:43.847898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d26648-b27f-44d4-bc90-d593a7434c10 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:43.883436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4cb8-250e-4dec-8eef-a2f72cb85150 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:43.899311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:43.899995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.900277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.900554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.910430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:43.914718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:43.921004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:43.924809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:43.925813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 156/301] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:43.932185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:43.955890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:43.956963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:43.957477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:43.957951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:43.973394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:43.978721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:43.980457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 146/302] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:43.982659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:43.986844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:44.119660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40088563-e18b-4bb5-814e-225e48cac69f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.165795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:44.166413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:44.166688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:44.166966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:44.167665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:44.167978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:44.169538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:44.179980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 157/303] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:44.180611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:44.186292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265f349-d659-4e5b-8b33-026eceabf6a2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.250847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:44.251816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:44.252106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:44.252451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:44.254047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:44.255780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:44.257811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 147/304] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:44.259100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:44.284298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:44.332998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858c570-6db7-4943-bb56-fbee76855a8f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.386109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:44.386900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:44.387354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:44.387802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:44.390553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:44.391072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:44.393537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:44.401737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:44.417893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 158/305] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:44.417893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:44.464292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc81b4-58b2-4322-9400-ab58ab39152d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.533507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:44.534558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:44.535216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:44.535667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] PATH_INFO: `/v3/users/b5c36c5e743d4d6eaeaee9ad92e94e60` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:44.561731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] RBAC: Authorizing `identity:get_user(user_id=b5c36c5e743d4d6eaeaee9ad92e94e60)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:44.563936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:44.573096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 148/306] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:44 2022] GET /identity/v3/users/b5c36c5e743d4d6eaeaee9ad92e94e60 => generated 293 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:44.573096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:44.611270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd05f0-ae15-4cad-bb0f-9856649e306b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.658386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:44.674964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:44.676018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:44.676507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:44.676956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] PATH_INFO: `/v3/groups` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:44.678144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:44.680949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:44.716421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695e21a-c18d-4f67-8cab-49e1aaa6a880 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.749237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:44.758651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:44.759299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:44.767106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:44.767106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] PATH_INFO: `/v3/users/5fcb0eae27e74ed295b197f3f34c06d8` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:44.767106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] RBAC: Authorizing `identity:get_user(user_id=5fcb0eae27e74ed295b197f3f34c06d8)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:44.767106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:44.767106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 149/307] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:44 2022] GET /identity/v3/users/5fcb0eae27e74ed295b197f3f34c06d8 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:44.767106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:44.808262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 159/308] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Aug 24 14:11:44 2022] POST /identity/v3/groups => generated 229 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:11:44.808931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:44.902068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391b05-6ef1-454d-9f7d-4d21498d4da5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.954148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630bc0c-c950-4a44-88cd-3728d5acb60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:44.955134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630bc0c-c950-4a44-88cd-3728d5acb60d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:44.955778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630bc0c-c950-4a44-88cd-3728d5acb60d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:44.956201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630bc0c-c950-4a44-88cd-3728d5acb60d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:44.957962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630bc0c-c950-4a44-88cd-3728d5acb60d None admin] RBAC: Adding query filter params (role.id=64e702b1f13d4b62988575b426f8f8a3, user.id=b5c36c5e743d4d6eaeaee9ad92e94e60, scope.system=all) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:44.958514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630bc0c-c950-4a44-88cd-3728d5acb60d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:44.960470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630bc0c-c950-4a44-88cd-3728d5acb60d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:44.968742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e06e1-34b1-4264-a14e-07ae9a092fce None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:44.969401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 150/309] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 24 14:11:44 2022] GET /identity/v3/role_assignments?user.id=b5c36c5e743d4d6eaeaee9ad92e94e60&scope.system=all&role.id=64e702b1f13d4b62988575b426f8f8a3 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:45.018373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ea1e0-8fec-4220-82f9-1876bce7356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:45.019074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea1e0-8fec-4220-82f9-1876bce7356e None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:45.019401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea1e0-8fec-4220-82f9-1876bce7356e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:45.019636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea1e0-8fec-4220-82f9-1876bce7356e None admin] PATH_INFO: `/v3/system/users/5fcb0eae27e74ed295b197f3f34c06d8/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:45.021319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ea1e0-8fec-4220-82f9-1876bce7356e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5fcb0eae27e74ed295b197f3f34c06d8, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:45.022839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ea1e0-8fec-4220-82f9-1876bce7356e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:45.136576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 160/310] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Aug 24 14:11:44 2022] PUT /identity/v3/system/users/5fcb0eae27e74ed295b197f3f34c06d8/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:45.430977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfc96a-f8a4-4c27-ba3a-09002c5e9eb5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:45.431636 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfc96a-f8a4-4c27-ba3a-09002c5e9eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:45.432104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfc96a-f8a4-4c27-ba3a-09002c5e9eb5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:45.434123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 151/311] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:45.441055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d76d93-c3e0-4cab-9553-98949f1d985c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:45.441701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d76d93-c3e0-4cab-9553-98949f1d985c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:45.442217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d76d93-c3e0-4cab-9553-98949f1d985c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:45.463288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d4d76d93-c3e0-4cab-9553-98949f1d985c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:45.464877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d4d76d93-c3e0-4cab-9553-98949f1d985c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:45.550099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:45.595593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 161/312] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:11:46.099206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfc96a-f8a4-4c27-ba3a-09002c5e9eb5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:46.234627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:46.305512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfc96a-f8a4-4c27-ba3a-09002c5e9eb5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:46.368989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:46.370110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:46.370560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:46.371012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:46.392375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:46.405614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:46.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:46.407423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 152/313] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:46.423181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d76d93-c3e0-4cab-9553-98949f1d985c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:46.611999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d76d93-c3e0-4cab-9553-98949f1d985c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:46.680614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:46.681565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:46.682068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:46.682578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:46.684072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:46.684618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:46.688134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:46.705955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 162/314] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:46.715918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:46.892439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afe67-962a-4438-a2b6-eac5842a83eb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:46.917321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:46.939672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fe6a2-355c-4f4f-bbef-bb210db00df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:46.940253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fe6a2-355c-4f4f-bbef-bb210db00df9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:46.940533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fe6a2-355c-4f4f-bbef-bb210db00df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:46.940807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fe6a2-355c-4f4f-bbef-bb210db00df9 None admin] PATH_INFO: `/v3/users/fdbb6920786d4f3a9318537b1decb75e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:46.942062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fe6a2-355c-4f4f-bbef-bb210db00df9 None admin] RBAC: Authorizing `identity:get_user(user_id=fdbb6920786d4f3a9318537b1decb75e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:46.943548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fe6a2-355c-4f4f-bbef-bb210db00df9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:46.945231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 153/315] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:46 2022] GET /identity/v3/users/fdbb6920786d4f3a9318537b1decb75e => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:46.955970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:47.046428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:47.059105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa064336-8052-4ef5-91a5-53ab54134084 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.059454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa064336-8052-4ef5-91a5-53ab54134084 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.059650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa064336-8052-4ef5-91a5-53ab54134084 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.061871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 154/316] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:47.067724 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d2740-a504-473a-b454-32db3086e7a5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.068035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d2740-a504-473a-b454-32db3086e7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.068274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d2740-a504-473a-b454-32db3086e7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.087806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f28d2740-a504-473a-b454-32db3086e7a5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:47.089061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f28d2740-a504-473a-b454-32db3086e7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:47.166740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207b94f-ebd2-427c-8f1a-b9da1adc9907 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:47.174140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 155/317] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:47.238395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:47.239340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.239828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.240291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.341925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc0d5d-05f6-48b0-9d10-a6c843774ba5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.342618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc0d5d-05f6-48b0-9d10-a6c843774ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.343099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc0d5d-05f6-48b0-9d10-a6c843774ba5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.344761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 156/318] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:47.350629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea2bb-46f5-4c48-94ce-3212c6128a59 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.351267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea2bb-46f5-4c48-94ce-3212c6128a59 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.351754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea2bb-46f5-4c48-94ce-3212c6128a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.373307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3cfea2bb-46f5-4c48-94ce-3212c6128a59 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:47.375166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3cfea2bb-46f5-4c48-94ce-3212c6128a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:47.421281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:47.425395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:47.456414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 157/319] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:47.463470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfea2bb-46f5-4c48-94ce-3212c6128a59 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:47.513830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:47.520820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:47.580683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfea2bb-46f5-4c48-94ce-3212c6128a59 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:47.621072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:47.621906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.622339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.622742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.624429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:47.626247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:47.628310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 158/320] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:11:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:47.634283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:47.653356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 163/321] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:47.655407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:47.750715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01e129-fbca-475b-a38d-195eadcd4afb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:47.771568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:47.796116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40984e19-9943-4ad0-aa7c-04556c3ee31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:47.797151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40984e19-9943-4ad0-aa7c-04556c3ee31a None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.797546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40984e19-9943-4ad0-aa7c-04556c3ee31a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.798063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40984e19-9943-4ad0-aa7c-04556c3ee31a None admin] PATH_INFO: `/v3/groups` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.799057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40984e19-9943-4ad0-aa7c-04556c3ee31a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:47.801703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40984e19-9943-4ad0-aa7c-04556c3ee31a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:47.823434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c16459-9fae-48ed-8569-dc9f8b434057 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:47.861735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:47.862413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.862697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.862974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.864214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:47.865674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:47.867343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 164/322] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:47 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:47.873965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:47.881139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:47.967683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 159/323] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Aug 24 14:11:47 2022] POST /identity/v3/groups => generated 226 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:11:47.969247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7d0a-3e38-43c1-ad48-cb2902030b1c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:47.969707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7d0a-3e38-43c1-ad48-cb2902030b1c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:47.970248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7d0a-3e38-43c1-ad48-cb2902030b1c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:47.974557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 160/324] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:47.974557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7d0a-3e38-43c1-ad48-cb2902030b1c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:47.979383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941b7f0-6fd5-4ab7-8845-70140761ab61 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:48.035052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d1439-38a3-42fd-b8f4-0559afd830ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:48.035636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1439-38a3-42fd-b8f4-0559afd830ae None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:48.035911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1439-38a3-42fd-b8f4-0559afd830ae None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:48.036184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1439-38a3-42fd-b8f4-0559afd830ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:48.050725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d1439-38a3-42fd-b8f4-0559afd830ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d44d1439-38a3-42fd-b8f4-0559afd830ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:48.057145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:48.058173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:48.058956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 165/325] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:48.060995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84330236-6768-4572-9342-9b4c46deae76 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:48.060995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84330236-6768-4572-9342-9b4c46deae76 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:48.060995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84330236-6768-4572-9342-9b4c46deae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:48.079162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-84330236-6768-4572-9342-9b4c46deae76 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:48.080434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-84330236-6768-4572-9342-9b4c46deae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:48.123880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7d0a-3e38-43c1-ad48-cb2902030b1c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:48.174494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:48.175583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:48.176205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:48.176486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:48.176755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:48.177495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] RBAC: Adding query filter params (role.id=64e702b1f13d4b62988575b426f8f8a3, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e, user.id=fdbb6920786d4f3a9318537b1decb75e) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:48.177797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:48.179422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:48.190062 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 166/326] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:48.190062 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84330236-6768-4572-9342-9b4c46deae76 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:48.190340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 161/327] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:11:47 2022] GET /identity/v3/role_assignments?user.id=fdbb6920786d4f3a9318537b1decb75e&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=64e702b1f13d4b62988575b426f8f8a3 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:48.339833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84330236-6768-4572-9342-9b4c46deae76 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:48.398230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:48.399756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:48.400550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:48.401351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:48.402992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:48.403847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:48.407201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:48.426654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 167/328] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:48.432026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:48.489902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:48.516742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:48.589405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:48.623630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768bcc-5af5-4056-9cbd-977b6b505b12 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:48.655271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c403c-1a11-4f83-8b83-f3208f1881a6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:48.667498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:48.668188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:48.668455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:48.668723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] PATH_INFO: `/v3/users/5fcb0eae27e74ed295b197f3f34c06d8` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:48.669983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] RBAC: Authorizing `identity:get_user(user_id=5fcb0eae27e74ed295b197f3f34c06d8)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:48.671516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:48.674158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 162/329] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:48 2022] GET /identity/v3/users/5fcb0eae27e74ed295b197f3f34c06d8 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:48.679676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:48.714836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:48.715824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:48.716376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:48.716856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:48.729395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:48.735098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:48.736097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:48.736919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 168/330] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:48.745301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:48.874737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd67b-f952-457e-9f9c-51d90ee30885 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:48.925327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:48.969664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:48.970276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:48.970551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:48.970816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:48.971771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] RBAC: Adding query filter params (role.id=80a1f8f62542401a855036270bf2c51a, user.id=5fcb0eae27e74ed295b197f3f34c06d8, scope.system=all) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:48.972073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:48.973813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:48.976079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a4b8-dabf-448c-b4a2-bfab408a76cd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:48.993222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 163/331] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 24 14:11:48 2022] GET /identity/v3/role_assignments?user.id=5fcb0eae27e74ed295b197f3f34c06d8&scope.system=all&role.id=80a1f8f62542401a855036270bf2c51a => generated 510 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:49.033392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:49.071882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:49.073386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:49.074224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:49.075197 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:49.077938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:49.079818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:49.082732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:49.102415 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 169/332] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:49.114186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:49.355971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f205584-348e-4fbf-9a4c-65adb74ecb95 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:49.436812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4c0da-f5f3-4489-ae42-6c0b5311f067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:49.437969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4c0da-f5f3-4489-ae42-6c0b5311f067 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:49.438608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4c0da-f5f3-4489-ae42-6c0b5311f067 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:49.439067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4c0da-f5f3-4489-ae42-6c0b5311f067 None admin] PATH_INFO: `/v3/users/b5c36c5e743d4d6eaeaee9ad92e94e60` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:49.441188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4c0da-f5f3-4489-ae42-6c0b5311f067 None admin] RBAC: Authorizing `identity:get_user(user_id=b5c36c5e743d4d6eaeaee9ad92e94e60)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:49.444243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4c0da-f5f3-4489-ae42-6c0b5311f067 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:49.447555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 164/333] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:49 2022] GET /identity/v3/users/b5c36c5e743d4d6eaeaee9ad92e94e60 => generated 293 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:49.458622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:49.674044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40c7ca-2aa4-457f-a5da-016319d799ca None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:49.746381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a83987-722a-4ada-b8b4-267de027b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:49.747454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83987-722a-4ada-b8b4-267de027b1d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:49.748022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83987-722a-4ada-b8b4-267de027b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:49.748451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83987-722a-4ada-b8b4-267de027b1d4 None admin] PATH_INFO: `/v3/system/users/b5c36c5e743d4d6eaeaee9ad92e94e60/roles/64e702b1f13d4b62988575b426f8f8a3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:49.750894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a83987-722a-4ada-b8b4-267de027b1d4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b5c36c5e743d4d6eaeaee9ad92e94e60, role_id=64e702b1f13d4b62988575b426f8f8a3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:49.753219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a83987-722a-4ada-b8b4-267de027b1d4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:49.832695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 170/334] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Aug 24 14:11:49 2022] PUT /identity/v3/system/users/b5c36c5e743d4d6eaeaee9ad92e94e60/roles/64e702b1f13d4b62988575b426f8f8a3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:11:51.083914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50492e-1c74-4c1f-948d-60b366352a55 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:51.084701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50492e-1c74-4c1f-948d-60b366352a55 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:51.084701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50492e-1c74-4c1f-948d-60b366352a55 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:51.086205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 165/335] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:51.099441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6972b4-733c-4075-b9bc-c3eed406a19f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:51.100177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6972b4-733c-4075-b9bc-c3eed406a19f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:51.100177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6972b4-733c-4075-b9bc-c3eed406a19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:51.128641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4a6972b4-733c-4075-b9bc-c3eed406a19f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:51.129924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4a6972b4-733c-4075-b9bc-c3eed406a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:51.198181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:51.242467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 171/336] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:51.530420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee696599-c2f5-4d54-8a27-3bb8c9f25d96 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:51.530903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee696599-c2f5-4d54-8a27-3bb8c9f25d96 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:51.531085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee696599-c2f5-4d54-8a27-3bb8c9f25d96 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:51.532456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 166/337] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:11:51.548330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8856-f7c7-4618-8bfe-ab5e1b34d755 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:51.548626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8856-f7c7-4618-8bfe-ab5e1b34d755 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:51.548950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8856-f7c7-4618-8bfe-ab5e1b34d755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:51.572970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-45ee8856-f7c7-4618-8bfe-ab5e1b34d755 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:51.574288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45ee8856-f7c7-4618-8bfe-ab5e1b34d755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:51.680819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 172/338] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:51.783609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee696599-c2f5-4d54-8a27-3bb8c9f25d96 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:51.827640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:51.917003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee696599-c2f5-4d54-8a27-3bb8c9f25d96 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:51.960011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:51.960602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:51.960879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:51.961248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:51.975695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:51.980755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:51.982607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:51.983467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 167/339] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:51.992526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8856-f7c7-4618-8bfe-ab5e1b34d755 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:52.140669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8856-f7c7-4618-8bfe-ab5e1b34d755 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:52.150895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:52.181661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:52.182327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:52.182602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:52.182867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:52.183552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:52.183952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:52.186135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:52.200618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 173/340] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:52.209659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:52.262193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:52.386431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:52.408202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e5865-4d1d-45dd-977f-1d989efe4556 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:52.439523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c213526-e24e-429c-abc4-970f50a469ff None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:52.483872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:52.484866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:52.485285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:52.485732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:52.489734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:52.490654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:52.491781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:52.493115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:52.524923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:52.530939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:52.532089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:52.533054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:52.535829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 174/341] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:11:52.541681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:52.607352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:52.683320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:52.685097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:52.758735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6899b4e-0c00-414c-9f43-6c2716491994 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:52.826233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:52.827228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:52.827658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:52.828163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:52.829100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:52.829692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:52.831370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:52.846266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 175/342] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:11:52.862773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:52.917005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:52.933322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 168/343] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:52.940217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:53.034401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:53.056931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58966c97-ae07-4a20-aaef-e0f9ac0b5fea None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:53.152783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:53.154383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:53.154865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:53.155336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:53.226623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9b4d2-f6a5-4797-a175-680ad202541f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:53.322863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409e94e-0c86-46a7-b51b-132585d94be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:53.323748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409e94e-0c86-46a7-b51b-132585d94be8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:53.324026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409e94e-0c86-46a7-b51b-132585d94be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:53.324301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409e94e-0c86-46a7-b51b-132585d94be8 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:53.325868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409e94e-0c86-46a7-b51b-132585d94be8 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:53.327366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409e94e-0c86-46a7-b51b-132585d94be8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:53.330212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 169/344] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:52 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:53.331546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6974f-fa39-4026-801d-efc908df1734 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:53.331822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6974f-fa39-4026-801d-efc908df1734 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:53.332090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6974f-fa39-4026-801d-efc908df1734 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:53.333554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 170/345] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:53.334816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291894d0-d174-481e-ac6d-d06879c095e4 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:53.335090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291894d0-d174-481e-ac6d-d06879c095e4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:53.335354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291894d0-d174-481e-ac6d-d06879c095e4 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:53.337078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 171/346] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:53.343454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868275a-ef51-422b-b1cb-2e20a44538bc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:53.343859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868275a-ef51-422b-b1cb-2e20a44538bc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:53.344022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868275a-ef51-422b-b1cb-2e20a44538bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:53.366665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2868275a-ef51-422b-b1cb-2e20a44538bc None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:53.367907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2868275a-ef51-422b-b1cb-2e20a44538bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:53.380469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:53.383729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:53.447413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:53.460950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 172/347] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:53.464766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc74ec-863f-48bf-a460-a7a4c7bc25c8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:53.465258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc74ec-863f-48bf-a460-a7a4c7bc25c8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:53.465721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc74ec-863f-48bf-a460-a7a4c7bc25c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:53.498995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e3fc74ec-863f-48bf-a460-a7a4c7bc25c8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:53.501258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e3fc74ec-863f-48bf-a460-a7a4c7bc25c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:53.606994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:53.632794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 176/348] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 771 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 14:11:53.634441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:53.674800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 173/349] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:53.676094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc74ec-863f-48bf-a460-a7a4c7bc25c8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:53.922788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e443e-0b14-4a23-b82c-cf248abfbfe2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:53.950365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc74ec-863f-48bf-a460-a7a4c7bc25c8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:53.979520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:53.996918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d9349-44b7-48aa-b9e4-7e6fda32cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:53.999957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d9349-44b7-48aa-b9e4-7e6fda32cad2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:53.999957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d9349-44b7-48aa-b9e4-7e6fda32cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:53.999957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d9349-44b7-48aa-b9e4-7e6fda32cad2 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.000524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d9349-44b7-48aa-b9e4-7e6fda32cad2 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:54.002384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d9349-44b7-48aa-b9e4-7e6fda32cad2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:54.004710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 174/350] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:53 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:54.006223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a3e2b-7aff-4f15-bfd7-b071e25210e9 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.006622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a3e2b-7aff-4f15-bfd7-b071e25210e9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.006984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a3e2b-7aff-4f15-bfd7-b071e25210e9 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.007370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:54.008686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.009481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.010267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 175/351] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:54.010626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.011108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a3e2b-7aff-4f15-bfd7-b071e25210e9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:54.023318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:54.026651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:54.030820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 177/352] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:11:53 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:54.032788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:54.131748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:54.201977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a3e2b-7aff-4f15-bfd7-b071e25210e9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:54.211127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defa676-3b92-42a9-88a5-427828822564 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:54.264713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d7732-81b4-4535-ad32-b667f07ed14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:54.266031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7732-81b4-4535-ad32-b667f07ed14c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.266335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7732-81b4-4535-ad32-b667f07ed14c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.266664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d7732-81b4-4535-ad32-b667f07ed14c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.280604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d7732-81b4-4535-ad32-b667f07ed14c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:54.283002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c4b77-d041-46dc-9629-c82446835b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:54.284029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c4b77-d041-46dc-9629-c82446835b9e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.284476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c4b77-d041-46dc-9629-c82446835b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.286109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c4b77-d041-46dc-9629-c82446835b9e None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-954d7732-81b4-4535-ad32-b667f07ed14c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:54.286343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:54.288156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:54.289917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:54.291551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c4b77-d041-46dc-9629-c82446835b9e None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:54.293749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 176/353] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:54.294648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d4265-3a92-41b3-bdc6-49567dc637e9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.294866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d4265-3a92-41b3-bdc6-49567dc637e9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.295149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d4265-3a92-41b3-bdc6-49567dc637e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.298932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c4b77-d041-46dc-9629-c82446835b9e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:54.298932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 178/354] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:11:54 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:54.300280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4621e5-d100-4d30-849c-32de28163d3f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.300759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4621e5-d100-4d30-849c-32de28163d3f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.301876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4621e5-d100-4d30-849c-32de28163d3f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.303827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 179/355] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:54.304949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4621e5-d100-4d30-849c-32de28163d3f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:54.325867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-629d4265-3a92-41b3-bdc6-49567dc637e9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:54.327911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-629d4265-3a92-41b3-bdc6-49567dc637e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:54.414939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:54.457365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 177/356] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:54.463177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d4265-3a92-41b3-bdc6-49567dc637e9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:54.481583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4621e5-d100-4d30-849c-32de28163d3f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:54.515517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:54.534424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:54.536410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.536765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.537202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.538743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:54.539285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:54.542097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:54.560951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 180/357] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:11:54 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:54.560951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:54.628318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d4265-3a92-41b3-bdc6-49567dc637e9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:54.672443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:54.673116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.673390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.673651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.687832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:54.693701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:54.695486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 178/358] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:54.697042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:54.752416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:54.791489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4b545-4088-419b-b0ea-eddfa42dfcbc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:54.830590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23316aab-3ecf-472d-ac2b-ba1b9e4739d4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:54.859121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:54.860672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.861439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.863451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.865189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:54.866089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:54.869090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:54.872849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fe58f-6fd1-4e42-b463-d8c0dae55664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:54.873487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe58f-6fd1-4e42-b463-d8c0dae55664 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.873763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe58f-6fd1-4e42-b463-d8c0dae55664 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.874065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe58f-6fd1-4e42-b463-d8c0dae55664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.874823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fe58f-6fd1-4e42-b463-d8c0dae55664 None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:54.875130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fe58f-6fd1-4e42-b463-d8c0dae55664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:54.876616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fe58f-6fd1-4e42-b463-d8c0dae55664 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:54.882938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 179/359] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:11:54 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=80a1f8f62542401a855036270bf2c51a => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:54.887436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a197-4d41-4ebb-9a35-e10aab4cccd3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:54.887751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a197-4d41-4ebb-9a35-e10aab4cccd3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:54.888099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a197-4d41-4ebb-9a35-e10aab4cccd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:54.890615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 181/360] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:54.892576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:54.934250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3bf0a197-4d41-4ebb-9a35-e10aab4cccd3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:54.936462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3bf0a197-4d41-4ebb-9a35-e10aab4cccd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:55.005709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:55.114509 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 180/361] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:55.115917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0a197-4d41-4ebb-9a35-e10aab4cccd3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:55.124872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37955893-de51-464d-85c5-6cc37626a46b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:55.189151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:55.250494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:55.252460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:55.253160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:55.253822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:55.255326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:55.256031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:55.259008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:55.279343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 182/362] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:55.281310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:55.346440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0a197-4d41-4ebb-9a35-e10aab4cccd3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:55.426184 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:55.427780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:55.428573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:55.429367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:55.549339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6066e8d-117a-4906-b8b7-d6deac226216 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:55.578506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:55.646641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:55.648754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:55.649275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:55.649979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:55.679947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:55.687894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:55.693290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:55.720557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:55.721549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 14:11:55.722383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:55.726122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 183/363] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:55.729216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:55.836808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:56.051138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 181/364] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 936 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:56.052597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:56.067498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c5ee3-0331-4684-b4aa-787fe9f3e5e0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:56.139391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:56.140420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:56.140909 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:56.141395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:56.239045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:56.291069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948a31-bb73-41ea-83a6-94c337c4ef74 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:56.314035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:56.317681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:56.347252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:56.348039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:56.348171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:56.348958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:56.349696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:56.350097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:56.351851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:56.354138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:56.376553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 182/365] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:56.376553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:56.488915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 184/366] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:56.489833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:56.516922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac29828-0ce1-4665-82cb-58f0831bf44a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:56.567047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:56.567718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:56.567996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:56.568321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:56.590451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:56.595856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:56.597577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 183/367] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:11:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:56.599473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:56.603258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:56.628597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccbe70-9508-4f45-a45a-3673b7faec0b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:56.674892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:56.675579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:56.675861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:56.676136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:56.677458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:56.678994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:56.680994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 185/368] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:56 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:56.681851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:56.755873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:56.821675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3291b3-b5ca-4470-9f4a-b89e91f70ebb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:56.856811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764d9f4-4cc5-43e4-8dd8-a193bcec356c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:56.865497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:56.866193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:56.866476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:56.866750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:56.867972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:56.869539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:56.871762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 186/369] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:56 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:56.878026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:56.890162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:56.943434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:56.944102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:56.944393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:56.944669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] PATH_INFO: `/v3/users/b5c36c5e743d4d6eaeaee9ad92e94e60` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:56.947121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] RBAC: Authorizing `identity:get_user(user_id=b5c36c5e743d4d6eaeaee9ad92e94e60)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:56.948717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:56.951912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 184/370] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:11:56 2022] GET /identity/v3/users/b5c36c5e743d4d6eaeaee9ad92e94e60 => generated 293 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:56.952858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:57.072604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247b1c8-421e-48ec-94f0-260c9e8ef6c4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:57.124376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:57.125417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.126046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.126534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.127432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:57.128021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:57.129645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:57.142098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 187/371] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:57.143553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:57.205212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909509-3208-4237-a857-33e2839a91c9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:57.251933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:57.256235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:57.279843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:57.281185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.281674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.282710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.285236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:57.289315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:57.294507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 185/372] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:11:56 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:57.296153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:57.307626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09680862-05b4-45be-99bd-c914c429df50 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:57.360709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:57.362312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.362965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.363442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.365179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:57.367653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:57.378038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 188/373] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:11:57 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:57.379362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:57.527112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cefce5-b66a-4879-a266-f06daedf6da3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:57.555092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6613da1-fe25-48e0-bb89-e6410796a6bb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:57.594432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:57.595064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.595338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.595596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.596442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] RBAC: Adding query filter params (role.id=64e702b1f13d4b62988575b426f8f8a3, user.id=b5c36c5e743d4d6eaeaee9ad92e94e60, scope.system=all) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:57.596755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:57.598417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:57.606121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4c0aa-a03a-47b0-b6c8-1c829287b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:57.607059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c0aa-a03a-47b0-b6c8-1c829287b811 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.607567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c0aa-a03a-47b0-b6c8-1c829287b811 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.607888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c0aa-a03a-47b0-b6c8-1c829287b811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.612319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:57.614980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 186/374] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 24 14:11:57 2022] GET /identity/v3/role_assignments?user.id=b5c36c5e743d4d6eaeaee9ad92e94e60&scope.system=all&role.id=64e702b1f13d4b62988575b426f8f8a3 => generated 510 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:57.615998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:57.681446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:57.706330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4c0aa-a03a-47b0-b6c8-1c829287b811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:57.709617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4c0aa-a03a-47b0-b6c8-1c829287b811 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:57.782781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e953239-f256-4b2c-999f-c731e3b61dce None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:57.845337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:57.846204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.846546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.848078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.848078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:57.848078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:57.849385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:57.856916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 187/375] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:11:57 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=80a1f8f62542401a855036270bf2c51a => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:57.858683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:57.923201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 189/376] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:11:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:11:57.927129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbc71f-8bca-477a-905d-afbe3f3b8acc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.928036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbc71f-8bca-477a-905d-afbe3f3b8acc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.928770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbc71f-8bca-477a-905d-afbe3f3b8acc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.932238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 190/377] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:57.935091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d5e7f-9609-4a19-8c8e-726dfac825f5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:57.936050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d5e7f-9609-4a19-8c8e-726dfac825f5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:57.936830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d5e7f-9609-4a19-8c8e-726dfac825f5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:57.942684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 191/378] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:11:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:11:57.950637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d5e7f-9609-4a19-8c8e-726dfac825f5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:58.036308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:58.047071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7497d3-01ab-4f98-b6d7-7b6ace337bf3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:58.073915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:58.124745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e929fc4-d54b-42a7-8156-91a6942bfb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:58.125384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e929fc4-d54b-42a7-8156-91a6942bfb9e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:58.125663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e929fc4-d54b-42a7-8156-91a6942bfb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:58.125935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e929fc4-d54b-42a7-8156-91a6942bfb9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:58.126747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e929fc4-d54b-42a7-8156-91a6942bfb9e None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:58.127054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e929fc4-d54b-42a7-8156-91a6942bfb9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:58.128551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e929fc4-d54b-42a7-8156-91a6942bfb9e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:58.135788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 188/379] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:11:57 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:58.138975 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c90d5-7084-42df-98c6-78efe16efe8f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:58.139332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c90d5-7084-42df-98c6-78efe16efe8f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:58.139678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c90d5-7084-42df-98c6-78efe16efe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:58.166898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4a7c90d5-7084-42df-98c6-78efe16efe8f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:58.169469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4a7c90d5-7084-42df-98c6-78efe16efe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:58.171417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d5e7f-9609-4a19-8c8e-726dfac825f5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:58.252370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba91348-d01f-4875-a1ac-9ac0a1112a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:58.253438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba91348-d01f-4875-a1ac-9ac0a1112a14 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:58.254127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba91348-d01f-4875-a1ac-9ac0a1112a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:58.254701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba91348-d01f-4875-a1ac-9ac0a1112a14 None admin] PATH_INFO: `/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:58.256726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba91348-d01f-4875-a1ac-9ac0a1112a14 None admin] RBAC: Authorizing `identity:get_project(project_id=f90d0e4c756a418fb562e2e8abdd6ef4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:58.259903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba91348-d01f-4875-a1ac-9ac0a1112a14 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:58.264489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 192/380] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:11:57 2022] GET /identity/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4 => generated 344 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:58.266476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c494e7-3582-49a8-9adc-7dbfaf8c367c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:58.266923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c494e7-3582-49a8-9adc-7dbfaf8c367c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:58.267357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c494e7-3582-49a8-9adc-7dbfaf8c367c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:58.301326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-15c494e7-3582-49a8-9adc-7dbfaf8c367c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:11:58.303710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-15c494e7-3582-49a8-9adc-7dbfaf8c367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:11:58.329491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 189/381] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:58.331771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c90d5-7084-42df-98c6-78efe16efe8f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:58.473672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 193/382] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:11:58.577942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c90d5-7084-42df-98c6-78efe16efe8f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:58.598249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:58.652349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:58.654018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:58.654534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:58.655265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] PATH_INFO: `/v3/groups/e651be899ef1438bbd8c2f9982615403` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:58.667923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] RBAC: Authorizing `identity:get_group(group_id=e651be899ef1438bbd8c2f9982615403)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:58.671410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:58.674832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 190/383] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:11:58 2022] GET /identity/v3/groups/e651be899ef1438bbd8c2f9982615403 => generated 229 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:58.684607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c494e7-3582-49a8-9adc-7dbfaf8c367c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:58.762355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:58.923799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c494e7-3582-49a8-9adc-7dbfaf8c367c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:58.973060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:59.033124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:59.034182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:59.039235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:59.039845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:59.041334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] RBAC: Adding query filter params (group.id=e651be899ef1438bbd8c2f9982615403, role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f90d0e4c756a418fb562e2e8abdd6ef4) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:59.041879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:59.044804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:59.069305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 194/384] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:11:58 2022] GET /identity/v3/role_assignments?group.id=e651be899ef1438bbd8c2f9982615403&scope.project.id=f90d0e4c756a418fb562e2e8abdd6ef4&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 260 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:59.071637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:59.355736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a872d-6cc1-4dc6-bf1b-7bc47ee57426 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:59.402237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:59.411409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620bd33-4da3-4faa-82e1-ae7e14b82d58 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:59.425361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:59.426928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:59.427722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:59.428483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:59.454122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:59.463431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:59.464980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:11:59.466828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:11:59.470926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 191/385] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:11:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:59.480206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:59.497962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:59.499146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:59.499505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:59.499959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:59.547199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:11:59.548358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:11:59.566262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:11:59.568458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:11:59.574421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 195/386] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:11:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 498 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:11:59.583386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:59.699285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce372-b5ec-4938-9c03-87fbb2a4b463 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:11:59.766265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:11:59.767604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:11:59.768271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:11:59.768934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:11:59.770316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:11:59.771039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:11:59.773718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:11:59.789907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 192/387] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:11:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:11:59.799263 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:11:59.951697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c79ce2-2b14-43e1-b133-210d5af00d0d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:00.002488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:00.021090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adfd55-18e1-4847-bf22-f20538bdb1b9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:00.060436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:00.061068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:00.061341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:00.061619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:00.062319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:00.062618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:00.065747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:00.083181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 196/388] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:11:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 501 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:00.091720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:00.111382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:00.112485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:00.112974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:00.113463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:00.119293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:00.122505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:00.128124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 193/389] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:11:59 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:00.143109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:00.189299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:00.457648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad290940-baa4-4e8e-a479-d14c836bdda6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:00.479834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e8469-ca07-4e2f-ab83-48c1569d3415 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:00.526321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:00.526791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:00.527190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:00.527656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:00.530119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:00.534095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:00.542234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 194/390] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:00 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:00.546744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:00.595516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:00.596307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:00.596440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:00.596868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:00.602903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:00.604431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:00.606651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 197/391] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:00 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:00.618488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:00.619120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:00.758852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c62496-00e7-4a9d-8446-39e3ecc14b2b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:00.833735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:00.834903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:00.835374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:00.835946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/groups/c52dc4d5d50546069174d3af529e1784/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:00.840022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] RBAC: Authorizing `identity:create_grant(project_id=aefb95afbe7845b5a5da1d55abf22a62, group_id=c52dc4d5d50546069174d3af529e1784, role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:00.843745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:00.885340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:00.900731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7347b1-15e9-4a7b-856a-fe971212ddde None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:00.949228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 195/392] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 24 14:12:00 2022] PUT /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/groups/c52dc4d5d50546069174d3af529e1784/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:12:00.957334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8c587-ba56-42a0-8fa9-b30bb6d528d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:00.958110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8c587-ba56-42a0-8fa9-b30bb6d528d4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:00.958401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8c587-ba56-42a0-8fa9-b30bb6d528d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:00.958675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8c587-ba56-42a0-8fa9-b30bb6d528d4 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:00.963015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8c587-ba56-42a0-8fa9-b30bb6d528d4 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:00.964640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8c587-ba56-42a0-8fa9-b30bb6d528d4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:00.974183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 198/393] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:00 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:00.984217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:01.133622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:01.146232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189279e-d87f-4b19-888f-b62c75c32413 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:01.221193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa19cb3-5b47-44f1-bae6-beeac40c8fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:01.221910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa19cb3-5b47-44f1-bae6-beeac40c8fe0 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:01.222197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa19cb3-5b47-44f1-bae6-beeac40c8fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:01.222479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa19cb3-5b47-44f1-bae6-beeac40c8fe0 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/groups/c52dc4d5d50546069174d3af529e1784/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:01.227782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa19cb3-5b47-44f1-bae6-beeac40c8fe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=aefb95afbe7845b5a5da1d55abf22a62, group_id=c52dc4d5d50546069174d3af529e1784, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:01.229413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa19cb3-5b47-44f1-bae6-beeac40c8fe0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:01.329113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 196/394] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 24 14:12:00 2022] PUT /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62/groups/c52dc4d5d50546069174d3af529e1784/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:12:01.380015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8394d-d858-4ead-88ab-41765aeabea3 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:01.380464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8394d-d858-4ead-88ab-41765aeabea3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:01.380935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8394d-d858-4ead-88ab-41765aeabea3 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:01.384031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 199/395] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:01.389838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5844fb-278e-40d9-bccc-3306b4ec0157 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:01.390272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5844fb-278e-40d9-bccc-3306b4ec0157 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:01.390498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5844fb-278e-40d9-bccc-3306b4ec0157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:01.423309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ac5844fb-278e-40d9-bccc-3306b4ec0157 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:01.424553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ac5844fb-278e-40d9-bccc-3306b4ec0157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:01.543228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 197/396] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:01.817743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5249-05a5-43d2-8a69-e953e1c892ff None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:01.818803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5249-05a5-43d2-8a69-e953e1c892ff None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:01.818803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5249-05a5-43d2-8a69-e953e1c892ff None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:01.820860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 200/397] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:01.829216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645579fd-3cb5-4b7c-bed9-b31adcf26afa None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:01.830135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645579fd-3cb5-4b7c-bed9-b31adcf26afa None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:01.830135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645579fd-3cb5-4b7c-bed9-b31adcf26afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:01.857152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-645579fd-3cb5-4b7c-bed9-b31adcf26afa None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:01.858580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-645579fd-3cb5-4b7c-bed9-b31adcf26afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:01.864326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b5249-05a5-43d2-8a69-e953e1c892ff None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:02.004130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b5249-05a5-43d2-8a69-e953e1c892ff None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:02.051279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:02.054765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:02.056942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:02.058290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:02.080354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 198/398] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:02.099506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:02.104358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1158b-45b2-44c7-8bb6-0dc7b7d790f7 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:02.105231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1158b-45b2-44c7-8bb6-0dc7b7d790f7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:02.105953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1158b-45b2-44c7-8bb6-0dc7b7d790f7 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:02.109295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 199/399] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:12:02.116362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca7294-321c-4bed-8674-bd9fcee0af54 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:02.117259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca7294-321c-4bed-8674-bd9fcee0af54 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:02.118026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca7294-321c-4bed-8674-bd9fcee0af54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:02.127446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:02.129118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:02.130671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:02.142147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 201/400] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:12:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:02.152384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:02.179101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-61ca7294-321c-4bed-8674-bd9fcee0af54 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:02.180330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-61ca7294-321c-4bed-8674-bd9fcee0af54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:02.193428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:02.214407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:02.308084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 200/401] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:12:02.381509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3719acb-c1f8-4dc5-908f-36137cb5fe5a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:02.428799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:02.429827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:02.430405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:02.430892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:02.431814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:02.432374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:02.434193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:02.445338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 202/402] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:02.452467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca7294-321c-4bed-8674-bd9fcee0af54 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:02.579927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca7294-321c-4bed-8674-bd9fcee0af54 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:02.621951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:02.623356 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:02.623852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:02.624330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:02.625980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:02.627733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:02.629653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 201/403] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:02 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:02.635701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:02.658791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:02.676041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:02.790960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:02.821286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81a812-d58e-4b2e-bfba-3c4318c89a95 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:02.885148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:02.885978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:02.886339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:02.886724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:02.888813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:02.890430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:02.892570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 203/404] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:02 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:02.899646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:02.922583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b8fb8-85bd-4e63-9f7b-6bc9c57120d5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:02.989938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:02.991122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:02.991122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:02.991389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:03.005647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:03.010950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:03.011979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:12:03.013054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:03.017791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 202/405] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:03.019337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:03.053405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c001a-80e8-4b5d-a589-d2bb3bb60ba4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:03.103245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:03.104301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:03.105099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:03.105588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:03.120651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:03.128437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:03.129507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:03.130466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:03.132833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 204/406] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:03.133207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:03.134171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:03.153886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:03.190026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dfcf2-7ef5-4f51-8902-467da4f4cee9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:03.240275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:03.241722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:03.242277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:03.242760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/groups/c52dc4d5d50546069174d3af529e1784/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:03.245487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13307a6663b4948b0439bb4ebc46e3e, group_id=c52dc4d5d50546069174d3af529e1784, role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:03.248443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:03.324061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b4596-5c77-4edd-9a8f-233cb76c47c5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:03.331583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 203/407] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 24 14:12:03 2022] PUT /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/groups/c52dc4d5d50546069174d3af529e1784/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:12:03.333322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:03.408242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:03.408783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:03.409062 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:03.410032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:03.413676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:03.414172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:03.415950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:03.429605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 205/408] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:12:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:03.436808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:03.586414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:03.651935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d476bc7-fcd7-465c-8d4c-cc9520c17b32 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:03.671425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db6ed-b5a7-4295-8936-7e52e76c5816 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:03.730667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:03.734250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:03.734808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:03.735249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:03.735359 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:03.737555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:03.740171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:03.743533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 206/409] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:03 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:03.750991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:03.756295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:03.757174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:03.757599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:03.758067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:03.758910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:03.759352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:03.760898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:03.772312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 204/410] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:03.779011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:03.992232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f27e2c-fc35-4f87-82b0-03c3151b1e2a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:04.029861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424b45-0982-4aa5-9ae7-d5f07b14ff05 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:04.097277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:04.098505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.099054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.099514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] PATH_INFO: `/v3/groups/e651be899ef1438bbd8c2f9982615403` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.105177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] RBAC: Authorizing `identity:get_group(group_id=e651be899ef1438bbd8c2f9982615403)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:04.113272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:04.116518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 205/411] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:03 2022] GET /identity/v3/groups/e651be899ef1438bbd8c2f9982615403 => generated 229 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:04.124320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:04.148035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:04.150669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.151466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.152750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.156068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:04.158146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:04.160736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 207/412] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:03 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:04.171242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:04.181719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:04.247568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:04.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4673e-31d4-4f72-a55a-fb67166b7691 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:04.343952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:04.345004 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.345490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.345964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] PATH_INFO: `/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.347770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] RBAC: Authorizing `identity:get_project(project_id=f90d0e4c756a418fb562e2e8abdd6ef4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:04.349565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:04.352191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 206/413] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:04 2022] GET /identity/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:04.364209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:04.491325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1a64-a3db-4608-915e-17e60853a59b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:04.524735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:04.536854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2690f-e0c5-447a-be8b-0f5ef9dbec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:04.537901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2690f-e0c5-447a-be8b-0f5ef9dbec6d None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.538383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2690f-e0c5-447a-be8b-0f5ef9dbec6d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.538819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2690f-e0c5-447a-be8b-0f5ef9dbec6d None admin] PATH_INFO: `/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4/groups/e651be899ef1438bbd8c2f9982615403/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.541487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2690f-e0c5-447a-be8b-0f5ef9dbec6d None admin] RBAC: Authorizing `identity:create_grant(project_id=f90d0e4c756a418fb562e2e8abdd6ef4, group_id=e651be899ef1438bbd8c2f9982615403, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:04.545397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2690f-e0c5-447a-be8b-0f5ef9dbec6d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:04.550686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890c085-b75d-40d5-b1b7-e38c66ac5fe0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:04.619726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc781f-bfc6-4ef0-b241-6864fe439b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:04.620889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc781f-bfc6-4ef0-b241-6864fe439b07 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.621379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc781f-bfc6-4ef0-b241-6864fe439b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.623808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc781f-bfc6-4ef0-b241-6864fe439b07 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/groups/c52dc4d5d50546069174d3af529e1784/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.627613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc781f-bfc6-4ef0-b241-6864fe439b07 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13307a6663b4948b0439bb4ebc46e3e, group_id=c52dc4d5d50546069174d3af529e1784, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:04.630476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc781f-bfc6-4ef0-b241-6864fe439b07 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:04.649624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 207/414] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 24 14:12:04 2022] PUT /identity/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4/groups/e651be899ef1438bbd8c2f9982615403/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:12:04.658080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5614f-5d27-455c-b7b7-cfb409ffe9c5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.658080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5614f-5d27-455c-b7b7-cfb409ffe9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.658080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5614f-5d27-455c-b7b7-cfb409ffe9c5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.660559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 208/415] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:04.671338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81c71-3b8b-4147-8885-609359266a97 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.673324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81c71-3b8b-4147-8885-609359266a97 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.673324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81c71-3b8b-4147-8885-609359266a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.707334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-83d81c71-3b8b-4147-8885-609359266a97 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:04.709616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-83d81c71-3b8b-4147-8885-609359266a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:04.760136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 208/416] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 24 14:12:04 2022] PUT /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e/groups/c52dc4d5d50546069174d3af529e1784/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:12:04.820897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906bef2-55f4-48c0-8679-f933963c4396 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.821130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906bef2-55f4-48c0-8679-f933963c4396 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.821456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906bef2-55f4-48c0-8679-f933963c4396 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.824454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 209/417] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:12:04.830437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4958-9b8e-4d9e-9f6a-09960d0ca4e5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:04.830723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4958-9b8e-4d9e-9f6a-09960d0ca4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:04.831010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4958-9b8e-4d9e-9f6a-09960d0ca4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:04.879598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-473a4958-9b8e-4d9e-9f6a-09960d0ca4e5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:04.880875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-473a4958-9b8e-4d9e-9f6a-09960d0ca4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:04.894916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 209/418] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:12:05.033201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:05.048895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 210/419] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:12:05.510646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d81c71-3b8b-4147-8885-609359266a97 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:05.580842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a4958-9b8e-4d9e-9f6a-09960d0ca4e5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:05.734843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d81c71-3b8b-4147-8885-609359266a97 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:05.755416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:05.797837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:05.798680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:05.798961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:05.799238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:05.820190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:05.826154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:05.827915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:05.829444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 210/420] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:05.837982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:05.840079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a4958-9b8e-4d9e-9f6a-09960d0ca4e5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:05.976596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:05.977603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:05.978039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:05.978504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:06.018167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:06.034009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:06.036360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:06.037874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:06.045458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 211/421] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:12:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 464 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:06.052216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:06.192688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e93fb9-dc30-45a1-8e44-680eb85b69b7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:06.235472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:06.265985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:06.266700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:06.266982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:06.267251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:06.267943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:06.268379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:06.269659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:06.291572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 211/422] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:12:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:06.300552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:06.334649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4132c31-a5e7-49ad-bbce-6d57748b5aea None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:06.398261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:06.437026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:06.437671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:06.437955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:06.438263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:06.439035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:06.439338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:06.440780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:06.470387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 212/423] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:06.479863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:06.517300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafcf13-efd5-42e5-a42b-59047aa25ea9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:06.626728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:06.627851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:06.628471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:06.628960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:06.783216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:06.784988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:06.791374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ef00d-0621-43a8-a601-fb4d12677f61 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:06.826797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:06.831383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:06.874348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06213146-591a-4cdd-a6d5-69e41b6dd68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:06.875009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06213146-591a-4cdd-a6d5-69e41b6dd68e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:06.875286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06213146-591a-4cdd-a6d5-69e41b6dd68e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:06.875561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06213146-591a-4cdd-a6d5-69e41b6dd68e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:06.899977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 212/424] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:06.906422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:07.031915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06213146-591a-4cdd-a6d5-69e41b6dd68e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:07.035373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06213146-591a-4cdd-a6d5-69e41b6dd68e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:07.078892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83cdd3-5128-40e6-bbc1-c924cd9339c8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:07.118900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052ead1-29df-436b-a85a-dddce8950664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:07.119554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052ead1-29df-436b-a85a-dddce8950664 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.119832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052ead1-29df-436b-a85a-dddce8950664 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.120104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052ead1-29df-436b-a85a-dddce8950664 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.121500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052ead1-29df-436b-a85a-dddce8950664 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:07.124244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052ead1-29df-436b-a85a-dddce8950664 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:07.126658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 213/425] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:06 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:07.127915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330ef4-afb3-441c-8302-e707ac0dea78 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.128246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330ef4-afb3-441c-8302-e707ac0dea78 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.128786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330ef4-afb3-441c-8302-e707ac0dea78 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.130584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 214/426] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:07.135385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3330ef4-afb3-441c-8302-e707ac0dea78 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:07.166668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:07.246032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 213/427] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 761 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:07.246032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9de34-54e6-4939-9d03-43c03e3fafa1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.246032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9de34-54e6-4939-9d03-43c03e3fafa1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.246032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9de34-54e6-4939-9d03-43c03e3fafa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.266568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1aa9de34-54e6-4939-9d03-43c03e3fafa1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:07.267833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1aa9de34-54e6-4939-9d03-43c03e3fafa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:07.284579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:07.336662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3330ef4-afb3-441c-8302-e707ac0dea78 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:07.394631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 214/428] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:07.398229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9de34-54e6-4939-9d03-43c03e3fafa1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:07.402260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:07.403548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.403884 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.404232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.405708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:07.408020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:07.409696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 215/429] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:07 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:07.423670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:07.639385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9de34-54e6-4939-9d03-43c03e3fafa1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:07.674711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:07.715700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaa262-c2cb-4c83-ab64-4b55be879ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:07.716706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaa262-c2cb-4c83-ab64-4b55be879ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.716994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaa262-c2cb-4c83-ab64-4b55be879ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.717273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaa262-c2cb-4c83-ab64-4b55be879ca3 None admin] PATH_INFO: `/v3/projects/aefb95afbe7845b5a5da1d55abf22a62` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.718638 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaa262-c2cb-4c83-ab64-4b55be879ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=aefb95afbe7845b5a5da1d55abf22a62)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:07.720884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaa262-c2cb-4c83-ab64-4b55be879ca3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:07.722726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 215/430] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:07 2022] GET /identity/v3/projects/aefb95afbe7845b5a5da1d55abf22a62 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:07.724314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b2203-5a8f-48d2-92d4-f57893bd125d None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.724595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b2203-5a8f-48d2-92d4-f57893bd125d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.724871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b2203-5a8f-48d2-92d4-f57893bd125d None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.730184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 216/431] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:07.731858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b2203-5a8f-48d2-92d4-f57893bd125d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:07.746764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6c4f4-9cf3-4ab2-8b79-45f1691e113c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:07.750484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:07.812860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:07.813880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.814346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.814739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.815902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:07.816399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:07.819121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:07.837134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 216/432] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:12:07 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=80a1f8f62542401a855036270bf2c51a => generated 611 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:07.837134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:07.881286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b2203-5a8f-48d2-92d4-f57893bd125d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:07.944461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d315e5-ff59-4f28-9721-89923f332114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:07.945538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d315e5-ff59-4f28-9721-89923f332114 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.946086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d315e5-ff59-4f28-9721-89923f332114 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.946570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d315e5-ff59-4f28-9721-89923f332114 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.969591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d315e5-ff59-4f28-9721-89923f332114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-77d315e5-ff59-4f28-9721-89923f332114 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:07.980347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:07.982123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:07.983201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:07.989446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 217/433] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:12:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:07.992063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76f65c-139c-4f1a-a518-1439be601292 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:07.992773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76f65c-139c-4f1a-a518-1439be601292 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:07.993294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76f65c-139c-4f1a-a518-1439be601292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:07.996836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b5dc2-b70a-4c81-917d-3b9de36eb6ae None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:08.032223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ab76f65c-139c-4f1a-a518-1439be601292 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:08.033493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ab76f65c-139c-4f1a-a518-1439be601292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:08.060473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:08.061984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:08.062486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:08.062954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:08.064994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:08.067203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:08.068675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 217/434] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:07 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:08.069566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:08.082548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:08.190304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 218/435] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:08.191625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76f65c-139c-4f1a-a518-1439be601292 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:08.197454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:08.274607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec0c71-eb09-4c13-92b9-7cd751a45c8c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:08.342358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5dae0-4875-494d-b966-b28b371353f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:08.344416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5dae0-4875-494d-b966-b28b371353f0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:08.344864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5dae0-4875-494d-b966-b28b371353f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:08.345317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5dae0-4875-494d-b966-b28b371353f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:08.346477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5dae0-4875-494d-b966-b28b371353f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:08.346931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5dae0-4875-494d-b966-b28b371353f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:08.349408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5dae0-4875-494d-b966-b28b371353f0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:08.371226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 218/436] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:08.376331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cc9f2-48a8-4d99-8871-beaa220e2648 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:08.376331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cc9f2-48a8-4d99-8871-beaa220e2648 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:08.376331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cc9f2-48a8-4d99-8871-beaa220e2648 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:08.376331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 219/437] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:08.382762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3ebd-2505-4b8b-9e16-3bf512294486 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:08.383106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3ebd-2505-4b8b-9e16-3bf512294486 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:08.383300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e3ebd-2505-4b8b-9e16-3bf512294486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:08.394623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76f65c-139c-4f1a-a518-1439be601292 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:08.421299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-000e3ebd-2505-4b8b-9e16-3bf512294486 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:08.423476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-000e3ebd-2505-4b8b-9e16-3bf512294486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:08.448836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:08.450244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:08.450368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:08.450682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:08.451914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=aefb95afbe7845b5a5da1d55abf22a62) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:08.452120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:08.453698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:08.469504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 219/438] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:12:08 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=aefb95afbe7845b5a5da1d55abf22a62&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 611 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:08.469504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:08.540843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:08.549647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 220/439] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:08.587601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e3ebd-2505-4b8b-9e16-3bf512294486 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:08.603892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:08.648777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70348af-d633-4649-907a-3bf24d41a6f9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:08.695801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:08.696433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:08.696705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:08.696982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:08.795506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e3ebd-2505-4b8b-9e16-3bf512294486 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:08.801448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:08.803889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:08.857362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:08.857362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:08.857362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:08.857889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:08.881052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:08.890725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:08.891829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 14:12:08.892732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:08.893808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 221/440] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:08.897607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:08.900740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:08.966132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 220/441] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:08.968169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:08.987956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:09.045698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93bb4e-4b86-4a31-ba0f-f318d212b243 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.084998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.085643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.085947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.086231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.097810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399b113-5fed-4dc7-ad06-8d912cfaf09f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.099598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:09.104503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:09.106377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 222/442] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:09.107977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:09.145657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.146582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.147053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.147541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.148908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] RBAC: Adding query filter params (name=member) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:09.149433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:09.152049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:09.165098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 221/443] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:12:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:09.167332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:09.217486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:09.254491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c15a3-fd72-4b76-a932-2aeb05824f52 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.285146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:09.312084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.312696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.312930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.313208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.314610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:09.316208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:09.318019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 223/444] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:09 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:09.318904 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:09.332932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f511e-5d7f-46c2-94be-f9e5acd29d54 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.380102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.380970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.381370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.381774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.382874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:09.383412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:09.385624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:09.406082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 222/445] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:09.406082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:09.459637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c79e2-d092-4825-b573-22200db2e922 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.513901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.514573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.514849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.515116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.559740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:09.578953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246678b8-91f9-48a7-bee2-d720385e3cc1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.583399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:09.598901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:09.600562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:09.636132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.636805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.637160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.637438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.638966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:09.640538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:09.642551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 223/446] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:09 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:09.643167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:09.781326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 224/447] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:09.781633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ff254-6cef-4d94-bbd2-7517eff16c45 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.781791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:09.819698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.820323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.820600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.820870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.862075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:09.915752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:09.917846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:09.922908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:09.943868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd625b3-7258-40e4-a35d-b22391cde3c7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:09.985767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:09.986412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:09.986683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:09.986948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:09.987685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:09.987993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:09.989445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:09.996142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 225/448] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:12:09 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:09.997059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:10.066317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 224/449] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:10.072634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:10.165554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:10.189476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ae541-0afe-4185-8809-0ddd7861f4b6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:10.237915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dd292-e967-4bbf-b691-6e2bd66e1b7e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:10.259778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:10.260760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:10.261237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:10.261714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] PATH_INFO: `/v3/projects/d13307a6663b4948b0439bb4ebc46e3e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:10.266653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] RBAC: Authorizing `identity:get_project(project_id=d13307a6663b4948b0439bb4ebc46e3e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:10.269371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:10.272641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 226/450] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:09 2022] GET /identity/v3/projects/d13307a6663b4948b0439bb4ebc46e3e => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:10.280646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:10.300970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:10.301634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:10.301913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:10.302221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] PATH_INFO: `/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:10.303455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] RBAC: Authorizing `identity:get_project(project_id=f90d0e4c756a418fb562e2e8abdd6ef4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:10.304917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:10.306704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 225/451] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:12:10 2022] GET /identity/v3/projects/f90d0e4c756a418fb562e2e8abdd6ef4 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:10.310919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:10.313313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:10.422666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b20c-d018-4604-bf13-0e853603c910 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:10.449177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e99d1-e272-4378-80b4-e2246a95ebbe None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:10.464310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:10.464976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:10.466022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:10.466656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] PATH_INFO: `/v3/groups/c52dc4d5d50546069174d3af529e1784` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:10.467635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] RBAC: Authorizing `identity:get_group(group_id=c52dc4d5d50546069174d3af529e1784)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:10.469097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:10.470864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 227/452] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:10 2022] GET /identity/v3/groups/c52dc4d5d50546069174d3af529e1784 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:10.478638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:10.500711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:10.512385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:10.512971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:10.513138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:10.513283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] PATH_INFO: `/v3/groups/e651be899ef1438bbd8c2f9982615403` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:10.514735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] RBAC: Authorizing `identity:get_group(group_id=e651be899ef1438bbd8c2f9982615403)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:10.516099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:10.517794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 226/453] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 24 14:12:10 2022] GET /identity/v3/groups/e651be899ef1438bbd8c2f9982615403 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:10.524300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:10.597439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:10.614760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd3504-e06a-480f-8063-2e8ee7180087 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:10.657483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fcd4b-856e-407b-93d8-7dc4d4ac11db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:10.658232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fcd4b-856e-407b-93d8-7dc4d4ac11db None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:10.658574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fcd4b-856e-407b-93d8-7dc4d4ac11db None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:10.658964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fcd4b-856e-407b-93d8-7dc4d4ac11db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:10.660105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fcd4b-856e-407b-93d8-7dc4d4ac11db None admin] RBAC: Adding query filter params (group.id=c52dc4d5d50546069174d3af529e1784, role.id=80a1f8f62542401a855036270bf2c51a, scope.project.id=d13307a6663b4948b0439bb4ebc46e3e) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:10.660481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fcd4b-856e-407b-93d8-7dc4d4ac11db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:10.662416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fcd4b-856e-407b-93d8-7dc4d4ac11db None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:10.670549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 228/454] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:12:10 2022] GET /identity/v3/role_assignments?group.id=c52dc4d5d50546069174d3af529e1784&scope.project.id=d13307a6663b4948b0439bb4ebc46e3e&role.id=80a1f8f62542401a855036270bf2c51a => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:10.698998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010164ed-7631-4a6d-9a0a-74bf9b849851 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:10.768698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dde9be-a18b-48ac-81b2-f7459037a36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:10.769779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde9be-a18b-48ac-81b2-f7459037a36c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:10.770331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde9be-a18b-48ac-81b2-f7459037a36c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:10.770822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde9be-a18b-48ac-81b2-f7459037a36c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:10.772244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde9be-a18b-48ac-81b2-f7459037a36c None admin] RBAC: Adding query filter params (group.id=e651be899ef1438bbd8c2f9982615403, role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f90d0e4c756a418fb562e2e8abdd6ef4) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:10.772796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde9be-a18b-48ac-81b2-f7459037a36c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:10.775690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde9be-a18b-48ac-81b2-f7459037a36c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:10.798038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 227/455] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 24 14:12:10 2022] GET /identity/v3/role_assignments?group.id=e651be899ef1438bbd8c2f9982615403&scope.project.id=f90d0e4c756a418fb562e2e8abdd6ef4&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:13.659358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d8094-c4c3-45c3-8396-0a84d089ee78 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:13.660403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d8094-c4c3-45c3-8396-0a84d089ee78 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:13.660905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d8094-c4c3-45c3-8396-0a84d089ee78 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:13.662964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 229/456] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:13.669188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb47157-5686-4992-b145-768a435eeda7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:13.670431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb47157-5686-4992-b145-768a435eeda7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:13.671017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb47157-5686-4992-b145-768a435eeda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:13.678521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33622fd-3aef-469e-a47d-c51dea123223 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:13.678844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33622fd-3aef-469e-a47d-c51dea123223 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:13.679067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33622fd-3aef-469e-a47d-c51dea123223 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:13.680711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 230/457] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:13.686681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db236b06-86d8-419d-8499-6459dec92412 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:13.686990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db236b06-86d8-419d-8499-6459dec92412 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:13.687227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db236b06-86d8-419d-8499-6459dec92412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:13.695803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bdb47157-5686-4992-b145-768a435eeda7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:13.697043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bdb47157-5686-4992-b145-768a435eeda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:13.723155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-db236b06-86d8-419d-8499-6459dec92412 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:13.724405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-db236b06-86d8-419d-8499-6459dec92412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:13.782736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:13.816825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 228/458] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:13.876179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 231/459] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:12:14.096333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e3ff2-ef5e-419f-ad9b-6a51f5d1708f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:14.097092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e3ff2-ef5e-419f-ad9b-6a51f5d1708f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:14.097582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e3ff2-ef5e-419f-ad9b-6a51f5d1708f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:14.099609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 229/460] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:14.105192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac33393-433f-455c-8b62-da1669f1b869 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:14.105506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac33393-433f-455c-8b62-da1669f1b869 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:14.105816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac33393-433f-455c-8b62-da1669f1b869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:14.138752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8ac33393-433f-455c-8b62-da1669f1b869 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:14.139977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8ac33393-433f-455c-8b62-da1669f1b869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:14.243805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 232/461] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:14.322682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e3ff2-ef5e-419f-ad9b-6a51f5d1708f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:14.375155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac33393-433f-455c-8b62-da1669f1b869 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:14.399189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:14.513738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e3ff2-ef5e-419f-ad9b-6a51f5d1708f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:14.559746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:14.560845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:14.561373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:14.561860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:14.576648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:14.578610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:14.596051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:14.597059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:14.597945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:14.597945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:14.597945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:14.599906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 230/462] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:14.601015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:14.610426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac33393-433f-455c-8b62-da1669f1b869 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:14.681043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:14.692893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:14.693595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:14.696536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:14.696536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:14.707540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:14.709674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:14.727699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:14.728689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:14.729570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:14.729570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:14.729570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:14.732977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 233/463] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:14.734222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:14.748755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557ef77-816a-46c7-8654-bf6d62b5a602 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:14.795627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:14.796868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:14.797399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:14.797894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:14.813109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:14.814964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:14.825157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:14.832409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:14.833385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:14.834279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:14.834279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:14.834279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:14.834279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 231/464] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:14.835198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:14.883243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bcb6a-7b9c-4d11-8d09-ec71305e30f2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:14.926778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5496d-2d8b-45d2-a5e5-913420406cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:14.927411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5496d-2d8b-45d2-a5e5-913420406cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:14.927687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5496d-2d8b-45d2-a5e5-913420406cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:14.927957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5496d-2d8b-45d2-a5e5-913420406cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:14.928619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5496d-2d8b-45d2-a5e5-913420406cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:14.928929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5496d-2d8b-45d2-a5e5-913420406cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:14.930413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5496d-2d8b-45d2-a5e5-913420406cb6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:14.952548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:14.957187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 234/465] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:14.958922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e582b6-6e8d-4d39-a474-254655ab8f5d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:14.959221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e582b6-6e8d-4d39-a474-254655ab8f5d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:14.959496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e582b6-6e8d-4d39-a474-254655ab8f5d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:14.961060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 235/466] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:14.961758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e582b6-6e8d-4d39-a474-254655ab8f5d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:14.966906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71c0a-5658-45e3-8607-096e9d0ae35c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:15.009817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:15.010742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.011200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.011878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.012577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:15.012920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:15.014388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:15.034782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 232/467] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:15.035733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:15.082916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e582b6-6e8d-4d39-a474-254655ab8f5d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:15.110645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:15.124291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a772173-cb2d-4309-a5b2-2f30ecdf883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:15.124925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a772173-cb2d-4309-a5b2-2f30ecdf883c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.125203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a772173-cb2d-4309-a5b2-2f30ecdf883c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.125476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a772173-cb2d-4309-a5b2-2f30ecdf883c None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.126668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a772173-cb2d-4309-a5b2-2f30ecdf883c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:15.126668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a772173-cb2d-4309-a5b2-2f30ecdf883c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:15.128037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a772173-cb2d-4309-a5b2-2f30ecdf883c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:15.145534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 236/468] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:15.145534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a72b3e-65ba-476e-ae01-11a544643d60 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.145534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a72b3e-65ba-476e-ae01-11a544643d60 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.145534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a72b3e-65ba-476e-ae01-11a544643d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.164266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-61a72b3e-65ba-476e-ae01-11a544643d60 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:15.165534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-61a72b3e-65ba-476e-ae01-11a544643d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:15.167949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061047f6-1b1e-40ee-a31b-73592c4db897 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:15.219188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:15.220187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.220679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.222063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.222063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:15.224024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:15.258755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 237/469] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:15.259685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a72b3e-65ba-476e-ae01-11a544643d60 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:15.357034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 233/470] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:12:15 2022] POST /identity/v3/users => generated 246 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:15.362527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:15.403173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:15.492457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a72b3e-65ba-476e-ae01-11a544643d60 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:15.515487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:15.547763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa594e7f-1e9d-4cdf-8762-54e1375da729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:15.548399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa594e7f-1e9d-4cdf-8762-54e1375da729 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.548970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa594e7f-1e9d-4cdf-8762-54e1375da729 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.549446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa594e7f-1e9d-4cdf-8762-54e1375da729 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.550430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa594e7f-1e9d-4cdf-8762-54e1375da729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:15.551961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa594e7f-1e9d-4cdf-8762-54e1375da729 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:15.592689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a26f8-1809-404f-b944-386fcad5c946 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:15.640898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 238/471] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:12:15 2022] POST /identity/v3/users => generated 248 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:15.645014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc07707-39cc-4ea4-8171-4ac56069287f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.645572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc07707-39cc-4ea4-8171-4ac56069287f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.646146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc07707-39cc-4ea4-8171-4ac56069287f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.649330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 239/472] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:15.650957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc07707-39cc-4ea4-8171-4ac56069287f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:15.659298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf02d7-5ced-402d-8e1e-16a00984c01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:15.659298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf02d7-5ced-402d-8e1e-16a00984c01e None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.659298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf02d7-5ced-402d-8e1e-16a00984c01e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.659298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf02d7-5ced-402d-8e1e-16a00984c01e None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.659638 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf02d7-5ced-402d-8e1e-16a00984c01e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:15.659638 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf02d7-5ced-402d-8e1e-16a00984c01e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:15.751969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 234/473] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:12:15 2022] POST /identity/v3/users => generated 248 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:15.762559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510081c9-aa59-4b42-af4a-3ecaf0152a41 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:15.763265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510081c9-aa59-4b42-af4a-3ecaf0152a41 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:15.763962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510081c9-aa59-4b42-af4a-3ecaf0152a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:15.825470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-510081c9-aa59-4b42-af4a-3ecaf0152a41 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:15.827935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-510081c9-aa59-4b42-af4a-3ecaf0152a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:15.928318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc07707-39cc-4ea4-8171-4ac56069287f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:15.968718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:15.989663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 235/474] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:16.010280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:16.012265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:16.012736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:16.013211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:16.035418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:16.037165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:16.060777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:16.062592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:16.064750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:16.064750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:16.064750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:16.064750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 240/475] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:16.073584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510081c9-aa59-4b42-af4a-3ecaf0152a41 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:16.241268 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510081c9-aa59-4b42-af4a-3ecaf0152a41 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:16.286517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:16.287558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:16.288136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:16.288621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:16.289561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:16.290203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:16.292033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:16.307107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 236/476] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:16.314869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:16.319952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:16.389553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:16.405711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:16.464544 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d3f02-04ba-4ddd-8592-06c6f4417f8a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:16.509976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:16.510647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:16.510987 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:16.511289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:16.513038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:16.514658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:16.555788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.sql.core [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 14:12:16.555788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 14:12:16.555788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [parameters: {'id': '6581cf3e8d6b4c2e9dbce16d1975eef0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 14:12:16.555788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81725) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Aug 24 14:12:16.577464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 14:12:16.579191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application yield resource Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application next(self.gen) Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise self.value Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application session.commit() Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 14:12:16.580893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.session.flush() Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application compat.raise_( Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 14:12:16.584430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 14:12:16.591954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application conn.query(q) Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application result.read() Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 14:12:16.600963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [parameters: {'id': '6581cf3e8d6b4c2e9dbce16d1975eef0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:16.604419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 14:12:16.610640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:16.611519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 237/477] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 24 14:12:16 2022] POST /identity/v3/roles => generated 144 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:16.611519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:16.626688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba41811-f670-4df6-9fde-e8a8fd63d2fc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:16.713131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:16.713845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:16.714170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:16.714446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:16.717606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:16.719160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:16.829181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d972354-dece-4d55-8685-3b296d4210d9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:16.847873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 241/478] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:12:16 2022] POST /identity/v3/users => generated 249 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:16.874107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:16.908025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71731c3-2a13-4a1b-b215-1c6b22b5ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:16.909378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71731c3-2a13-4a1b-b215-1c6b22b5ff73 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:16.911988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71731c3-2a13-4a1b-b215-1c6b22b5ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:16.911988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71731c3-2a13-4a1b-b215-1c6b22b5ff73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:16.942071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71731c3-2a13-4a1b-b215-1c6b22b5ff73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c71731c3-2a13-4a1b-b215-1c6b22b5ff73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:16.964153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:16.975599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:16.977602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 238/479] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:12:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:16.979264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:17.127898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:17.157928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05ab99-6618-4ce6-995d-2f7fdd5ead33 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:17.215802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4865a-adca-415a-b9c6-6ba4d905a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:17.219395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4865a-adca-415a-b9c6-6ba4d905a64c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:17.219912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4865a-adca-415a-b9c6-6ba4d905a64c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:17.220405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4865a-adca-415a-b9c6-6ba4d905a64c None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:17.221740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4865a-adca-415a-b9c6-6ba4d905a64c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:17.222343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4865a-adca-415a-b9c6-6ba4d905a64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:17.225133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4865a-adca-415a-b9c6-6ba4d905a64c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:17.239055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 242/480] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:17.783329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef9b6-1212-4ed9-8837-99d890768266 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:17.784067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef9b6-1212-4ed9-8837-99d890768266 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:17.784306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef9b6-1212-4ed9-8837-99d890768266 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:17.785933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 239/481] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:17.793542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e62a0-d13e-4249-98c7-56d45f875787 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:17.794943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e62a0-d13e-4249-98c7-56d45f875787 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:17.795442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e62a0-d13e-4249-98c7-56d45f875787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:17.841206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-166e62a0-d13e-4249-98c7-56d45f875787 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:17.843612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-166e62a0-d13e-4249-98c7-56d45f875787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:18.001963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 243/482] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:12:18.346558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaef9b6-1212-4ed9-8837-99d890768266 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:18.524127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaef9b6-1212-4ed9-8837-99d890768266 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:18.578873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:18.579549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:18.579825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:18.580101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:18.596554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:18.601485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:18.603724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:18.605895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:18.607794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 240/483] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:18.614039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e62a0-d13e-4249-98c7-56d45f875787 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:18.761983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:18.837125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e62a0-d13e-4249-98c7-56d45f875787 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:18.916030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665fec8-65d5-40e7-9dc0-84fdcfefcfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:18.917007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665fec8-65d5-40e7-9dc0-84fdcfefcfec None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:18.917481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665fec8-65d5-40e7-9dc0-84fdcfefcfec None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:18.918616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665fec8-65d5-40e7-9dc0-84fdcfefcfec None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:18.919914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665fec8-65d5-40e7-9dc0-84fdcfefcfec None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:18.920526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665fec8-65d5-40e7-9dc0-84fdcfefcfec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:18.923166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665fec8-65d5-40e7-9dc0-84fdcfefcfec None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:18.944880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 244/484] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:18.950934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:19.052983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:19.154672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cbd28-16f4-4e0d-8fe1-a52e2c18cdff None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:19.175657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a6cb3-702c-4e34-b267-05a73b11bb98 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.175657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a6cb3-702c-4e34-b267-05a73b11bb98 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.175657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a6cb3-702c-4e34-b267-05a73b11bb98 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.175657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 245/485] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:19.186024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79810ac-03a7-499a-8210-243265183d6c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.186024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79810ac-03a7-499a-8210-243265183d6c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.186024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79810ac-03a7-499a-8210-243265183d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.209564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:19.210339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.212604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.212604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.233606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d79810ac-03a7-499a-8210-243265183d6c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:19.234985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d79810ac-03a7-499a-8210-243265183d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:19.248512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:19.250330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:19.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:19.276962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:19.278843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:19.278843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:19.278843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:19.278843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 241/486] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:19.286640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:19.395769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 246/487] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:12:19.452372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:19.462816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008d5f7-8f82-4877-a6dd-abe08c4e9412 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:19.513068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef8988-ee0e-444f-88f4-0216e751c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:19.513716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef8988-ee0e-444f-88f4-0216e751c4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.514018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef8988-ee0e-444f-88f4-0216e751c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.514299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef8988-ee0e-444f-88f4-0216e751c4c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.514982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef8988-ee0e-444f-88f4-0216e751c4c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:19.515293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef8988-ee0e-444f-88f4-0216e751c4c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:19.516762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef8988-ee0e-444f-88f4-0216e751c4c8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:19.535766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 242/488] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:19.541249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79810ac-03a7-499a-8210-243265183d6c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:19.626689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6233a-6ed5-4cbb-ab91-9e6506207da7 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.626977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6233a-6ed5-4cbb-ab91-9e6506207da7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.627419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6233a-6ed5-4cbb-ab91-9e6506207da7 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.629188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 243/489] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:19.631452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:19.634819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875cb7-7fd5-4177-bcf9-ea6adbdda79c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.635125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875cb7-7fd5-4177-bcf9-ea6adbdda79c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.635531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875cb7-7fd5-4177-bcf9-ea6adbdda79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.671106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f4875cb7-7fd5-4177-bcf9-ea6adbdda79c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:19.672366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f4875cb7-7fd5-4177-bcf9-ea6adbdda79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:19.741204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79810ac-03a7-499a-8210-243265183d6c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:19.820347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:19.821139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.821417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.821691 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.831092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:19.832575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:19.842811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 244/490] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:19.844705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e62d4-4928-44e2-8da7-24912ad53001 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.845239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e62d4-4928-44e2-8da7-24912ad53001 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.845707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e62d4-4928-44e2-8da7-24912ad53001 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.847634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 245/491] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:19.857874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:19.859021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:19.859911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:19.860864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 247/492] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:12:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:19.862508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaf2e5-c27e-498f-9305-90bbed2dab48 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:19.862968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaf2e5-c27e-498f-9305-90bbed2dab48 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:19.863362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaf2e5-c27e-498f-9305-90bbed2dab48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:19.869871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:19.895301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-dbcaf2e5-c27e-498f-9305-90bbed2dab48 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:19.896929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dbcaf2e5-c27e-498f-9305-90bbed2dab48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:20.047218 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7d69-72c0-4520-8b01-254369772ee4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:20.070041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 246/493] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:20.070041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaf2e5-c27e-498f-9305-90bbed2dab48 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:20.082416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:20.100689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:20.107251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c72dc-2180-4682-a556-748972fa8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:20.108271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c72dc-2180-4682-a556-748972fa8204 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:20.108736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c72dc-2180-4682-a556-748972fa8204 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:20.109200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c72dc-2180-4682-a556-748972fa8204 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:20.110635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c72dc-2180-4682-a556-748972fa8204 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:20.111150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c72dc-2180-4682-a556-748972fa8204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:20.113745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c72dc-2180-4682-a556-748972fa8204 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:20.180553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 248/494] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:12:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:20.181590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753730b-6307-43f3-a6ed-b2aee3f01666 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:20.181888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753730b-6307-43f3-a6ed-b2aee3f01666 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:20.182197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753730b-6307-43f3-a6ed-b2aee3f01666 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:20.185912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 249/495] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:20.191023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753730b-6307-43f3-a6ed-b2aee3f01666 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:20.260810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaf2e5-c27e-498f-9305-90bbed2dab48 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:20.341928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:20.342192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:20.342728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:20.346050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:20.369549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:20.377420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:20.379277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 247/496] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:20.380768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:20.410261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753730b-6307-43f3-a6ed-b2aee3f01666 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:20.477480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aa510-b712-4c77-9ba2-33332f3dd974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:20.478611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aa510-b712-4c77-9ba2-33332f3dd974 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:20.479105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aa510-b712-4c77-9ba2-33332f3dd974 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:20.479562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aa510-b712-4c77-9ba2-33332f3dd974 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:20.501038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aa510-b712-4c77-9ba2-33332f3dd974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:20.503548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f83aa510-b712-4c77-9ba2-33332f3dd974 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:20.510742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:20.512256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:20.513056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:20.514334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 250/497] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:20.516755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73884-d40a-4d18-976b-37861ea8eb3e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:20.517251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73884-d40a-4d18-976b-37861ea8eb3e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:20.525755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73884-d40a-4d18-976b-37861ea8eb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:20.550335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e6d73884-d40a-4d18-976b-37861ea8eb3e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:20.552410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e6d73884-d40a-4d18-976b-37861ea8eb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:20.561034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e597be-b8dc-454a-9b62-4d27a50f4342 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:20.601986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:20.627381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:20.628435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:20.628903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:20.629399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:20.657170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 251/498] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:20.657622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d73884-d40a-4d18-976b-37861ea8eb3e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:20.752811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:20.756565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:20.845325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d73884-d40a-4d18-976b-37861ea8eb3e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:20.867375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:20.904391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:20.905394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:20.905869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:20.906412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:20.907686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:20.908255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:20.911114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:20.923935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 252/499] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:20.927764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:20.936217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 248/500] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:20.937586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:20.995180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:21.147565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc55c-9b07-460d-972f-a7f42ba5ee3e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.159314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b881ce4-3ddd-40db-a533-4caf1e0d8c74 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.212634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.213478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.213829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.214247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.214624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.215394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.215542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.215945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.216489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:21.216869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:21.218240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:21.227929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 253/501] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:21.228831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.229614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:21.237252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:21.243621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 249/502] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:21.244417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.288732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:21.354952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:21.367148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9df3e-e9da-4cee-b212-444cd0740dbc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.379042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24af7c-aadc-4bd6-8f24-0a973afbb05f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.412111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.413138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.413650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.414296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.427581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.428390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.428664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.428931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.433715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:21.435400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:21.440712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:21.442216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:21.453712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:21.455845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:21.457633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:21.457633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.457633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:21.457633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 254/503] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:21.457633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:21.462565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:21.463523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:21.464345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:21.464345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.464345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:21.465167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 250/504] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:21.467079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.571044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:21.596708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930a27e-a137-4eb2-958c-5fd3eda67eca None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.596926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b728e-7c2a-4f72-8bcc-7236fb1f24e4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.638658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.639433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.639807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.640161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.643737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.644860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.645381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.645887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.652929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:21.654468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:21.659522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:21.661020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:21.664666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:21.671398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:21.679588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:21.681278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:21.681278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.681278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:21.681278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 251/505] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:21.681278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:21.681812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:21.684296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:21.686841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:21.686841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:21.686841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:21.686841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 255/506] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:21.686841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.802005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112301-8ed4-4d56-b339-d507551d73ca None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.802786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95596cfb-05da-44f5-90f2-577dd06e14a7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.842228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.842853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.843128 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.843393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.844073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:21.844372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:21.845906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:21.846123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:21.846690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:21.846973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:21.847265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:21.847920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:21.848224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:21.849176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:21.849633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:21.857148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 252/507] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:21.858098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.865198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 256/508] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:21.866121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:21.941977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:21.990566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1277c-4b2b-4635-bc6f-369df8448356 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:21.995728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f53ca2-e2f1-498f-8701-fda5a92bfc10 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.033064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.033945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.034264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.034538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.035182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:22.035479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.036900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.055110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 257/509] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:22.055110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.055384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.055837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.056317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.056801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.058064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:22.059348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.061911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.083487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 253/510] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:22.085023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.165300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:22.170338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda145e-aeab-42ff-ab69-486fcf7778f5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.191735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:22.211523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.212077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.212360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.212631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.213291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:22.213597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.214175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670fb4e-68bd-4397-80d5-708603f93ecf None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.215221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.228611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 258/511] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:22.229820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.256080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.256906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.257170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.257438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.269832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.271365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:22.287388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:22.288377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:22.289238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:22.289238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:22.289238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:22.289238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 254/512] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:22.290128 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.361433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a9bc8-becc-4486-8b8c-9e2d28e598e9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.404217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.404922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.405210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.405486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.414491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.416014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:22.428914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:22.430342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 259/513] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:22.432575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.443269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4655773-d5f0-4931-b028-1ecc2d6ea2af None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.463752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:22.473960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:22.504698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.505368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.505648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.505924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.516380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.519290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:22.545116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:22.547077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 255/514] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:22.548796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.557955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb66b4a-3d0d-410b-bfbc-551a5298b78a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.602535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.603462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.603863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.604260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.616118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.618706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:22.639563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:22.641338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:22.643133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 260/515] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:22.644269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.677512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7597465-e385-4094-9759-5d429c2539e5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.717966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.718506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.718780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.719048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.719729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.721226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.780278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:22.787384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e13c-c2b1-48d2-94c3-11a016a9a147 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:22.826353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 256/516] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:12:22 2022] POST /identity/v3/users => generated 247 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:22.827524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.831132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:22.832216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:22.832706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:22.833171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:22.834473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:22.835014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:22.837590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:22.864779 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 261/517] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:22.867003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:22.892437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:23.044117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8038c3-ebd4-407b-84e2-6465c03e324b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:23.045395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb068fa2-8627-47ba-8d4a-e9911db74f8a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:23.107209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:23.108300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:23.108706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:23.108895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:23.109214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:23.109848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:23.110329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:23.110626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:23.110923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:23.111149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:23.112029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:23.112503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:23.113849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:23.114535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:23.150102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 262/518] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:23.150102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:23.173602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:23.187864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 257/519] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 24 14:12:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:23.187864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:23.336476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9fc8b-952a-4569-8e9f-c1943504f221 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:23.349067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:23.383320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362f1c1-5418-4d5b-b9c1-6f8464c511bc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:23.406601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:23.408025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:23.409026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:23.409765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:23.411345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:23.412143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:23.415244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:23.430063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:23.431001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:23.431281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:23.431547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:23.442040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:23.443580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:23.454489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 263/520] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:23.456550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:23.458973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:23.460577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 258/521] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:23.462370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:23.583935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:23.613895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a4906-a3d2-459b-aa9e-c5c7bccb1c3e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:23.646579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b9c32-34c7-4b96-b22c-4284646bf85c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:23.657048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:23.657681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:23.657952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:23.658322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:23.717552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:23.718481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:23.718481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:23.718752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:23.742757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:23.764458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:23.766062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:23.808205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:23.811029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:23.888995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:23.944407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 264/522] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:23.944407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:23.988188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 259/523] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:23.994050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:24.063233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:24.109580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea620a-5952-4ed6-86c8-52b92f5b7b5a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:24.147950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087a34e-e0f1-4d31-b4d7-b9f72cac4b55 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:24.164135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:24.165239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:24.165731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:24.166260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:24.167782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=613e985e94c54913a5e6b93367157fc0) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:24.168263 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:24.171152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:24.184502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 265/524] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:23 2022] GET /identity/v3/role_assignments?user.id=613e985e94c54913a5e6b93367157fc0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:24.184502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:24.207242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:24.210185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:24.210185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:24.210185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:24.211306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:24.211998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:24.214992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:24.257000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 260/525] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 24 14:12:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:24.259048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:24.280966 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:24.422625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44f7ab-6ed5-44da-826a-6fb83599a290 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:24.495645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704c58b-8bc6-4063-ad3b-7d26d8cadbf4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:24.499621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:24.508057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:24.509462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:24.510158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:24.510831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:24.537061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:24.540241 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:24.571244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:24.572539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:24.573225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:24.573877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:24.581036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:24.583091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:24.585014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:24.585014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:24.585014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:24.588143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 266/526] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:24.590709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:24.603303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:24.606620 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:24.639893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:24.641744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:24.643842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:24.643842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:24.643842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:24.644819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 261/527] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:24.646444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:24.805416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:24.816829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d66837-eecb-49b4-b78c-1241ca725978 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:24.843878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96f0b5-00e5-438f-8e31-342e72e40276 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:24.889114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f47f0-251b-4b2f-9561-3983bad4fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:24.890243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f47f0-251b-4b2f-9561-3983bad4fc49 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:24.890719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f47f0-251b-4b2f-9561-3983bad4fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:24.891209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f47f0-251b-4b2f-9561-3983bad4fc49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:24.920679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:24.921686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:24.922212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:24.922700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:24.923911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:24.924654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:24.927683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:24.959406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 262/528] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:24.962428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:24.997193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:25.039780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f47f0-251b-4b2f-9561-3983bad4fc49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:25.042506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f47f0-251b-4b2f-9561-3983bad4fc49 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:25.137856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c680de6-3928-4be5-990d-bae5446c57f6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:25.186777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:25.187571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.188011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.188609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.189391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:25.189831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:25.191571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:25.218542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 263/529] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:25.218542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:25.219602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:25.234589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 267/530] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:25.236823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7dfda-a11a-4eb8-9f95-f638d6f1a7f6 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.237409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7dfda-a11a-4eb8-9f95-f638d6f1a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.237947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7dfda-a11a-4eb8-9f95-f638d6f1a7f6 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.240219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 268/531] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:25.248479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7dfda-a11a-4eb8-9f95-f638d6f1a7f6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:25.364964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:25.410906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a3808-fd72-4ea1-9202-8e4aaa0c95ea None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:25.419764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7dfda-a11a-4eb8-9f95-f638d6f1a7f6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:25.454056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f929ee-4911-47f9-a038-4220c1ab7c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:25.454709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f929ee-4911-47f9-a038-4220c1ab7c32 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.454998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f929ee-4911-47f9-a038-4220c1ab7c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.455438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f929ee-4911-47f9-a038-4220c1ab7c32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.463512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f929ee-4911-47f9-a038-4220c1ab7c32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:25.466294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f929ee-4911-47f9-a038-4220c1ab7c32 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:25.467961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:25.468613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.468889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.469157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.477224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:25.478769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e0f929ee-4911-47f9-a038-4220c1ab7c32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:25.490741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:25.491655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:25.492516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:25.503179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:25.504255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:25.505321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 269/532] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:25.505902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 264/533] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:25.505983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:25.506100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fb9f-dde9-4875-8025-37c3992d981b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.506100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fb9f-dde9-4875-8025-37c3992d981b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.506100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fb9f-dde9-4875-8025-37c3992d981b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.530093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-cc88fb9f-dde9-4875-8025-37c3992d981b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:25.532238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cc88fb9f-dde9-4875-8025-37c3992d981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:25.576449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:25.634882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 265/534] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:25.634882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88fb9f-dde9-4875-8025-37c3992d981b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:25.672995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223adab-fa1c-4ce9-9980-b01a20c358b7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:25.721374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:25.722066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.722356 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.723101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.738396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:25.740088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:25.746631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:25.758783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:25.759950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:25.760940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:25.760940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:25.760940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:25.765721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 270/535] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:25.766683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:25.822812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88fb9f-dde9-4875-8025-37c3992d981b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:25.882088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:25.882729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.882992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.883352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.884093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:25.884972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:25.887473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:25.911217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 266/536] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:25.912118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:25.933562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5dc54-9491-490e-a9c7-97e64088f8e4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:25.936740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:25.982832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:25.983734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:25.984166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:25.984598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:25.985895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:25.986532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:25.988327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:26.005391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 271/537] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:26.006661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:26.083318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9b15d-d854-4836-b690-cb8da8c963d8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:26.113746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:26.126679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:26.127436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:26.127782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:26.128063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:26.128739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:26.129046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:26.130548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:26.149259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 267/538] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:26.149259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:26.172932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e966a1-1d4a-4fca-9959-cc4a225d5ce5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:26.236374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:26.236916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:26.237193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:26.237465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:26.238250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=9daf5be8a3dc48e78294882c62b8e60f) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:26.238561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:26.240065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:26.250221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 272/539] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:26 2022] GET /identity/v3/role_assignments?user.id=9daf5be8a3dc48e78294882c62b8e60f&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:26.251729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:26.292140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068635b-681f-42f2-85e2-90bf29200717 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:26.310587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:26.353056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:26.354323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:26.354856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:26.356119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:26.375653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:26.383612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:26.384586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:26.385378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:26.388798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 268/540] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:26.395572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:26.461096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:26.471234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11aede-8f35-4f7a-bcb0-7a5f00b98c23 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:26.533212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8a3b5-0067-4fc4-bd9b-d40c917d6922 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:26.539924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:26.541104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:26.541745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:26.542448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:26.544567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=aec389ead5164e778a9ee73e7ba7fc50) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:26.545194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:26.548157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:26.564480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 273/541] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:26 2022] GET /identity/v3/role_assignments?user.id=aec389ead5164e778a9ee73e7ba7fc50&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:26.564480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:26.621127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:26.622253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:26.622738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:26.623221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:26.651683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:26.655065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:26.695412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:26.696326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:26.697200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:26.699266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 269/542] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:26.705695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:26.749740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:26.763136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69a9c-7d34-4264-902d-8facc55e9da6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:26.838537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:26.839081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:26.840033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:26.840653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:26.841589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:26.841888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:26.843322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:26.859542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 274/543] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:26.865743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:26.956239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064693e3-b265-450c-9b3b-cb08dfc1c716 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:26.962111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:27.034144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:27.034776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:27.035046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:27.035310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:27.036021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:27.036323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:27.037842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:27.066166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 270/544] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:27.075127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:27.076094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a218f1c-8473-4852-b7bb-a907614e9894 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:27.124251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:27.125931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:27.128065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:27.128526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:27.152799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:27.156520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:27.180039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:27.181288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:27.182422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:27.182422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:27.182422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:27.185405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 275/545] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:27.191616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:27.196410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:27.323070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc4351-29cb-4875-a0ab-b6eaec2bfe83 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:27.348220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878098d0-0713-404d-8635-f5abeafb5334 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:27.380794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:27.380794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:27.380794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:27.380794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:27.381486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:27.381486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:27.381486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:27.391873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 276/546] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:27.397822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:27.460702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27509b-c52c-4ce3-91d2-bb2eed638371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:27.461361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27509b-c52c-4ce3-91d2-bb2eed638371 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:27.461813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27509b-c52c-4ce3-91d2-bb2eed638371 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:27.462191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27509b-c52c-4ce3-91d2-bb2eed638371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:27.463234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27509b-c52c-4ce3-91d2-bb2eed638371 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=48364ccf19434c4c9c09b4e5ce4bfc7b) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:27.463780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27509b-c52c-4ce3-91d2-bb2eed638371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:27.466621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27509b-c52c-4ce3-91d2-bb2eed638371 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:27.494881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 271/547] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:27 2022] GET /identity/v3/role_assignments?user.id=48364ccf19434c4c9c09b4e5ce4bfc7b&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 259 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:27.495835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:27.503186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ba5b1-7b8f-48fc-9736-a876c897b30a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:27.503644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ba5b1-7b8f-48fc-9736-a876c897b30a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:27.504108 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ba5b1-7b8f-48fc-9736-a876c897b30a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:27.512977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 272/548] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:12:27.517094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e75af2-0af7-4393-bc44-662ea4e6f4d1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:27.517555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e75af2-0af7-4393-bc44-662ea4e6f4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:27.518145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e75af2-0af7-4393-bc44-662ea4e6f4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:27.573060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-99e75af2-0af7-4393-bc44-662ea4e6f4d1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:27.575577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-99e75af2-0af7-4393-bc44-662ea4e6f4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:27.586979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506049b-54a2-4ef2-b4c2-42908382e12e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:27.638747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:27.684961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:27.686125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:27.686608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:27.687013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:27.701908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:27.708753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:27.738895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:27.740582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:27.743697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 277/549] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:27.751501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:27.786787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 273/550] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:28.016607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d034356-6c76-4e58-9aec-36c47424fc9d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:28.081365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:28.106809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:28.107862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:28.108337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:28.108815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:28.110190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:28.110703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:28.113353 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:28.164962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 278/551] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 24 14:12:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:28.171635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e75af2-0af7-4393-bc44-662ea4e6f4d1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:28.385790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e75af2-0af7-4393-bc44-662ea4e6f4d1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:28.442857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:28.512375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:28.513444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:28.513943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:28.514469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:28.588533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63a694-9678-413c-99a0-6567d32c1f4c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:28.601913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:28.667344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:28.669674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:28.676835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:28.677504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:28.677786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:28.678084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:28.701811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:28.713672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:28.714832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:28.715657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:28.716469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 279/552] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:28.724371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:28.795579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 274/553] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:28.801464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:28.861074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:28.874289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:28.923753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97722ae-5464-428f-8419-6b92c27def4c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:28.965958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:28.967145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:28.967650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:28.968136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:28.984714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:28.986837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:29.006138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:29.007170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:29.008158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:29.008158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:29.008158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:29.010849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 275/554] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:29.017380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:29.048002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f220b-7e03-4f92-ba7f-e3d141006446 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:29.100186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:29.101226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.101894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.102401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.103790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:29.104351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:29.106368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:29.118439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 280/555] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:29.127158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:29.214817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38cded-44a3-44e9-ad29-e155f1c4092c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:29.229389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:29.259437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:29.260452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.260937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.262257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.263224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:29.263739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:29.265373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:29.282028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 276/556] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:29.288051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:29.470063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc04ea-751b-42b6-9af3-8c91541df6df None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:29.561658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:29.563308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.564096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.564917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.573069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5874b-1eed-4429-b4d5-442ff2a0db1f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:29.578450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:29.591469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:29.595097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:29.627691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:29.629092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:29.629974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:29.632760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 277/557] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:29.639878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:29.686730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:29.688116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.689109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.690427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.704542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:29.715409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:29.723239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:29.754191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:29.755235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:29.756198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:29.756198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:29.756198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:29.758552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 281/558] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 632 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:29.769868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:29.858703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0953-8ea5-43f5-8886-f41a433c8d9d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:29.924864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09ed3f-b698-452e-8957-06040d7746ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:29.925700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ed3f-b698-452e-8957-06040d7746ab None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.925863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ed3f-b698-452e-8957-06040d7746ab None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.926165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ed3f-b698-452e-8957-06040d7746ab None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.926932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09ed3f-b698-452e-8957-06040d7746ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:29.927246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09ed3f-b698-452e-8957-06040d7746ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:29.928740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09ed3f-b698-452e-8957-06040d7746ab None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:29.950045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 278/559] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:29.950045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0efd-da70-40a6-b278-4b4b27532a9c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.950045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0efd-da70-40a6-b278-4b4b27532a9c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.950045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca0efd-da70-40a6-b278-4b4b27532a9c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.952533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 279/560] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:29.952533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ed4a2-3741-4e5b-bdf3-93e0eca0119d None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.952853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ed4a2-3741-4e5b-bdf3-93e0eca0119d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.953145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ed4a2-3741-4e5b-bdf3-93e0eca0119d None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.955998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 280/561] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:29.958794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee34e5-83c4-4ef3-856c-924a671b594a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:29.959068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee34e5-83c4-4ef3-856c-924a671b594a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:29.959365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee34e5-83c4-4ef3-856c-924a671b594a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:29.982487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7dee34e5-83c4-4ef3-856c-924a671b594a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:29.983648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7dee34e5-83c4-4ef3-856c-924a671b594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:30.000525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d90fe-bb2e-430f-9e79-b4545da4c765 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:30.060619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:30.061254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.061532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.061806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.062589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:30.063152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:30.064339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:30.082520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 282/562] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:30.085690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:30.109593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 281/563] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 14:12:30.111438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d687d-e038-4ce5-8709-04e1327bc70b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.111873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d687d-e038-4ce5-8709-04e1327bc70b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.112297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d687d-e038-4ce5-8709-04e1327bc70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.130801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:30.150034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a60d687d-e038-4ce5-8709-04e1327bc70b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:30.153067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a60d687d-e038-4ce5-8709-04e1327bc70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:30.179530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:30.278123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 282/564] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:30.278123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d687d-e038-4ce5-8709-04e1327bc70b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:30.297339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdd2ac-a66c-4a83-a285-d338e4f9052d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:30.367812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6a57b-fee8-4c7a-a20a-04b70132d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:30.369280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6a57b-fee8-4c7a-a20a-04b70132d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.369923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6a57b-fee8-4c7a-a20a-04b70132d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.372897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6a57b-fee8-4c7a-a20a-04b70132d1c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.374483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6a57b-fee8-4c7a-a20a-04b70132d1c7 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=36c4eeb768c04233bcbfc786b3ba20b3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:30.375296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6a57b-fee8-4c7a-a20a-04b70132d1c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:30.382300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6a57b-fee8-4c7a-a20a-04b70132d1c7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:30.402975 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 283/565] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:30 2022] GET /identity/v3/role_assignments?user.id=36c4eeb768c04233bcbfc786b3ba20b3&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:30.405696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93e7a-f4d5-4aef-807e-cad2bf5e37e1 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.406295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93e7a-f4d5-4aef-807e-cad2bf5e37e1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.406844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93e7a-f4d5-4aef-807e-cad2bf5e37e1 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.410128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 284/566] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:30.411776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a93e7a-f4d5-4aef-807e-cad2bf5e37e1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:30.474100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d687d-e038-4ce5-8709-04e1327bc70b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:30.547735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e232-985a-406b-9d17-5faa751f76a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:30.548888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e232-985a-406b-9d17-5faa751f76a1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.549439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e232-985a-406b-9d17-5faa751f76a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.549964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e232-985a-406b-9d17-5faa751f76a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.557060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:30.565097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0e232-985a-406b-9d17-5faa751f76a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:30.570299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0e232-985a-406b-9d17-5faa751f76a1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-5bf0e232-985a-406b-9d17-5faa751f76a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:30.583440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:30.584441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:30.585367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:30.587960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 283/567] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:12:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:30.589599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398635b-68d7-41da-8f97-a8dd2de5846a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.590148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398635b-68d7-41da-8f97-a8dd2de5846a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.590587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398635b-68d7-41da-8f97-a8dd2de5846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.605476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a93e7a-f4d5-4aef-807e-cad2bf5e37e1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:30.613223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1398635b-68d7-41da-8f97-a8dd2de5846a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:30.614901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1398635b-68d7-41da-8f97-a8dd2de5846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:30.618740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:30.686374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:30.687450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.688043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.688355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.708622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:30.713948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:30.715512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:30.716482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:30.723899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 284/569] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:30.723899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398635b-68d7-41da-8f97-a8dd2de5846a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:30.724326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 285/569] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:30.725507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:30.852408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398635b-68d7-41da-8f97-a8dd2de5846a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:30.892932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:30.894065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:30.894564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:30.895054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:30.896067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:30.896641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:30.898367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:30.910296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:30.925924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 285/570] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:12:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:30.925924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:30.964742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f385e-9aa1-4845-b738-cf428353e7fe None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.035016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.036648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.037517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.038319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.039925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:31.040887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:31.044118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:31.047845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333be6b0-c89b-489a-9ced-2258ab9e35e5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.061488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 286/571] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:31.063471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.069311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:31.123377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.125661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.126408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.127221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.154789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:31.166963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:31.169146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 286/572] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:31.172138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.210702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d839c0a-5e92-4c14-a683-ef36b15a0672 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.257684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.258357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.258651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.258900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.272539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:31.274158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:31.280447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:31.297719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:31.299021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:31.299964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:31.299964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:31.299964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:31.299964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 287/573] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:31.300767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.351027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255a636-c537-49e6-a938-9a4de0e02ab6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.403143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:31.422240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.423079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.423681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.424064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.440509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:31.442822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:31.460507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1331bd-6263-4bbe-8f6a-7e1e7f91960d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:31.463848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:31.465958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:31.467081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:31.467081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:31.467081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:31.467081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 287/574] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:31.471263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.516326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.517376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.517856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.518366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.519632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:31.520166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:31.522798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:31.535010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 288/575] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:31.535719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.608753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92bafb-f962-4117-9fb3-49227b792fed None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.621099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:31.660531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.661364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.661823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.662333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.670340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c99d4-e0dc-4c70-8fb0-7063aefe16ff None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.681192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:31.689760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:31.690845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:31.691818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:31.692685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:31.693330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 288/576] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:31.694391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.717686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.718818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.719329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.719807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.720679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:31.721208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:31.722881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:31.739407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 289/577] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:31.740659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.826316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61072e-a580-45fb-9dd3-7a77dfd8831c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.870797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.871718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.872378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.872803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.874160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:31.874640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:31.876529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:31.892704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 289/578] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:31.894159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:31.909402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7832f-9683-47d3-9f66-33d1f6465458 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:31.945362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:31.963856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:31.965012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:31.965772 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:31.966231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:31.985699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:31.987931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:32.013133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:32.014399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:32.015723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:32.015723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:32.015723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:32.015723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 290/579] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:32.015723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.045831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724588-1146-4798-984a-5f28b29656c8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.054081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:32.104058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.104856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.105689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.105956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.106696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:32.106996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.108414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.125897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 290/580] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:32.125897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.149339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24230b4e-34dd-4366-ae66-3e7e72f1f198 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.190377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.191022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.191297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.191568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.200235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.201640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:32.213952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:32.215782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:32.216770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 291/581] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:32.217541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.293660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:32.310203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d747f39-da21-4ae2-bcc9-8dded877f88a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.366326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:32.378943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.380137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.382711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.382711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.391825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aebae-822a-46df-a846-9a6c63d79454 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.397069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.398630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:32.419831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:32.421673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 14:12:32.423706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:32.426056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 291/582] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:32.427145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.452182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.453189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.453726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.454320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.455585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:32.456213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.458971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.490921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 292/583] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:32.492500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.562609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aa990-ab6e-4fa7-a422-ae1b8c7f933c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.614796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.615441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.615716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.615983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.629276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.630822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.631074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b5ee1-2632-406e-a500-90af05fbba74 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:32.648141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:32.659059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:32.661084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:32.661084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:32.661084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:32.661084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 292/584] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:32.661084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.661084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:32.686932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.687986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.688477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.688972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.690320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:32.690886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.693625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.718071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 293/585] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:32.718071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.738549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:32.796788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910ae0c-b0f0-470a-a544-7fce8ac9b7c9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.851841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9d7d6-e5ec-474d-9224-9900bcb79b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.852963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9d7d6-e5ec-474d-9224-9900bcb79b3b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.853447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9d7d6-e5ec-474d-9224-9900bcb79b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.853937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9d7d6-e5ec-474d-9224-9900bcb79b3b None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.855949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9d7d6-e5ec-474d-9224-9900bcb79b3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:32.856485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9d7d6-e5ec-474d-9224-9900bcb79b3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.859248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9d7d6-e5ec-474d-9224-9900bcb79b3b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.876275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711decc7-b89a-4571-9556-285f1d0818a0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:32.896222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 293/586] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:32.898866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556dd4c-4320-4b80-a7e9-8a93990dacc8 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.899614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556dd4c-4320-4b80-a7e9-8a93990dacc8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.900277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556dd4c-4320-4b80-a7e9-8a93990dacc8 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.910221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 294/587] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:32 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:32.912298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556dd4c-4320-4b80-a7e9-8a93990dacc8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:32.923994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:32.924661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:32.924940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:32.925214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:32.935648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:32.937261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:32.951393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:32.953227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 294/588] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:32.955144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:33.047669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:33.064814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:33.103150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556dd4c-4320-4b80-a7e9-8a93990dacc8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:33.125685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451b0c-66a1-4031-8e2d-1a048173544b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:33.158910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:33.159893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:33.160373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:33.161665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:33.162808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:33.163516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:33.166400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:33.186023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3178e-f530-438d-a63e-49f488ea1a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:33.187069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3178e-f530-438d-a63e-49f488ea1a39 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:33.187555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3178e-f530-438d-a63e-49f488ea1a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:33.187998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3178e-f530-438d-a63e-49f488ea1a39 None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/613e985e94c54913a5e6b93367157fc0/roles/adf17858daf74c2ea9cc93775172b567` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:33.189150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 295/589] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:33.190344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:33.222398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3178e-f530-438d-a63e-49f488ea1a39 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=613e985e94c54913a5e6b93367157fc0, role_id=adf17858daf74c2ea9cc93775172b567)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:33.225081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3178e-f530-438d-a63e-49f488ea1a39 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:33.330087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 295/590] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:12:32 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/613e985e94c54913a5e6b93367157fc0/roles/adf17858daf74c2ea9cc93775172b567 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:12:33.332367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd1e93-5ac7-461e-b024-273f115a9dc6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:33.333084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd1e93-5ac7-461e-b024-273f115a9dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:33.333552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd1e93-5ac7-461e-b024-273f115a9dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:33.367928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bfbd1e93-5ac7-461e-b024-273f115a9dc6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:33.368109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6113e2-e5d5-458e-a2fd-af489ab56d00 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:33.369270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bfbd1e93-5ac7-461e-b024-273f115a9dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:33.413329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:33.428432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:33.429275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:33.429548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:33.429829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:33.443813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:33.445484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:33.465123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:33.467281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:33.469358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:33.469358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:33.469358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:33.469358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 296/591] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:33.469358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:33.485540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 296/592] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:33.486930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd1e93-5ac7-461e-b024-273f115a9dc6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:33.594041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:33.605385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f9cfe-6cb6-4d10-bc06-6b6285833226 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:33.651325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:33.653197 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:33.653754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:33.654249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:33.655282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:33.655787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:33.657553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:33.679284 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 297/593] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 24 14:12:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:33.680490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:33.714932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd1e93-5ac7-461e-b024-273f115a9dc6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:33.721742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:33.775412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:33.776223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:33.776503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:33.777354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:33.786664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:33.788171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:33.803524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:33.804522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 14:12:33.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:33.810203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 297/594] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:33.811502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:33.838776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815205e7-0b2a-44b0-a5ea-46903b75ea8c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:33.878444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:33.878997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:33.879276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:33.879564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:33.880224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:33.880531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:33.881949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:33.894700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 298/595] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:33.895601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:33.975275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54d4a-b69f-4eb6-a053-d69adcb6e436 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:33.993022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:34.038721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.039981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.040429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.040882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.059713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:34.061259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:34.067905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:34.083318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53a6ee-06f7-4066-9445-58d960702609 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:34.086117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:34.087116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:34.088076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:34.088076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:34.088076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:34.088076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 298/596] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:34.089126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.141963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.142712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.143074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.143364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.158989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:34.164910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:34.165920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:34.167088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 299/597] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:34.169937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.242947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499003a5-80e9-405f-985c-1cbaab6ab2f4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.299922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.301103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.301575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.302229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.303599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:34.304340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:34.306015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:34.306194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9a6c-a6ae-4398-bc21-b1b9ccd56bce None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.338288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 299/598] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:34.340023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.366381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.367123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.367470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.368070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.373302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:34.381096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:34.384201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:34.398609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:34.399871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 300/599] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:34.401367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.425552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:34.484346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab789a6-7ee7-41c7-b19b-ccafa8e26df4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.527701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.528727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.529570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.529963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.531146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:34.531750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:34.533379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:34.549352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 300/600] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:34.554673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.558924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b539ead-1ec3-4c23-b6bd-5a92d5030382 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.607410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.608036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.608310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.608653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.609398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:34.609704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:34.611169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:34.626048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 301/601] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:34.626048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.700424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:34.721744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:34.726239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507046eb-19be-42f0-af32-cf708b8c1e2c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.770738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.770963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1343f0-5ae4-44f7-9d66-c534cb728b85 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.771549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.771895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.772313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.788559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:34.791257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:34.816083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:34.816844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:34.817230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:34.817630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:34.818327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:34.820268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:34.822234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:34.822234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:34.822234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:34.822550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:34.822550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:34.822550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:34.830240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 301/602] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:34.832220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.839450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 302/603] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:34.839450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:34.961853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679abfb4-7c87-4dae-87a5-6f7b7fa024a0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:34.971953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00aeed-881c-490d-b847-8d5074d6f24b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:35.012919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:35.022214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.027725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.027725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.027725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.032776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.033916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.034239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.034738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.035761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.037540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:35.037745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.049280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.050983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:35.051657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:35.052667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:35.053678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:35.059381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 302/604] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:35.059860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:35.077367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:35.079632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:35.082966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:35.082966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:35.082966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:35.082966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 303/605] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:35.082966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:35.202506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f1c28-4c35-49aa-84bb-be017bdd8082 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:35.239308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75111188-6308-4f80-abd7-22bceb41b265 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:35.254236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.254962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.255254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.255537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.256325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:35.256644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.258076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.277300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 303/606] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:35.277300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:35.294830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.295883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.296195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.296431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/aec389ead5164e778a9ee73e7ba7fc50/roles/adf17858daf74c2ea9cc93775172b567` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.315561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=aec389ead5164e778a9ee73e7ba7fc50, role_id=adf17858daf74c2ea9cc93775172b567)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.317053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.341849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:35.396003 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 304/607] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:12:35 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/aec389ead5164e778a9ee73e7ba7fc50/roles/adf17858daf74c2ea9cc93775172b567 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:12:35.398449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:35.421191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a3806-4fdd-499e-bd09-50dd2144b99e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:35.469487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:35.478516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.479290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.479711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.480289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.481985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:35.482879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.484628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.511785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 304/608] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:35.511785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:35.608265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a63fe-4b5b-4ecd-baa5-106d09ad9a5f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:35.662845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.663527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.663838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.664243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.665030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:35.665483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.667949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.684155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 305/609] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:35.685033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:35.694429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8a4f-f6b3-412d-ae00-6f5972c66125 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:35.718979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:35.767505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a41a8-e150-4433-8a7b-0ae9214ebcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.768510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a41a8-e150-4433-8a7b-0ae9214ebcda None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.768988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a41a8-e150-4433-8a7b-0ae9214ebcda None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.769507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a41a8-e150-4433-8a7b-0ae9214ebcda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.783840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a41a8-e150-4433-8a7b-0ae9214ebcda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.786774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a41a8-e150-4433-8a7b-0ae9214ebcda None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-cd2a41a8-e150-4433-8a7b-0ae9214ebcda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:35.812499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:35.814307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:35.816195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 305/610] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:35.817372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc5d7a-5bef-4281-a0d4-d2b15ec841aa None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.817859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc5d7a-5bef-4281-a0d4-d2b15ec841aa None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.818372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc5d7a-5bef-4281-a0d4-d2b15ec841aa None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.821152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 306/611] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:35.822579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc5d7a-5bef-4281-a0d4-d2b15ec841aa None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:35.834501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:35.843850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb91dd-5a51-4b53-aec5-3916d6c3e44f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:35.890021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:35.891078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:35.891500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:35.892065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/9daf5be8a3dc48e78294882c62b8e60f/roles/adf17858daf74c2ea9cc93775172b567` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:35.922503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=9daf5be8a3dc48e78294882c62b8e60f, role_id=adf17858daf74c2ea9cc93775172b567)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:35.925392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:35.977082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc5d7a-5bef-4281-a0d4-d2b15ec841aa None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:36.026372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00cef0-dd33-4cff-98e9-52da5df3013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:36.028027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00cef0-dd33-4cff-98e9-52da5df3013c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:36.028502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00cef0-dd33-4cff-98e9-52da5df3013c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:36.029044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00cef0-dd33-4cff-98e9-52da5df3013c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:36.046478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 306/612] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:12:35 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/9daf5be8a3dc48e78294882c62b8e60f/roles/adf17858daf74c2ea9cc93775172b567 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:12:36.046478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:36.046878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00cef0-dd33-4cff-98e9-52da5df3013c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:36.046878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00cef0-dd33-4cff-98e9-52da5df3013c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-7c00cef0-dd33-4cff-98e9-52da5df3013c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:36.069808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:36.070841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:36.071791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:36.078441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 307/613] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:36.080730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aeb71-54fa-462b-8c60-56e081defad6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:36.081209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aeb71-54fa-462b-8c60-56e081defad6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:36.081693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aeb71-54fa-462b-8c60-56e081defad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:36.116544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2a9aeb71-54fa-462b-8c60-56e081defad6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:36.118205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2a9aeb71-54fa-462b-8c60-56e081defad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:36.170681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:36.215300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105ab84-dda2-42a5-9179-f77e0845f3c0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:36.259250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 308/614] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:36.266621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:36.267571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:36.267898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aeb71-54fa-462b-8c60-56e081defad6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:36.268239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:36.268731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:36.269713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:36.270247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:36.271949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:36.289161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 307/615] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:36.300463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:36.311647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:36.453715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36fad1-fd1c-4252-9b30-f40f7d2a0672 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:36.492972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aeb71-54fa-462b-8c60-56e081defad6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:36.494545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:36.495773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:36.496289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:36.496818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/48364ccf19434c4c9c09b4e5ce4bfc7b/roles/adf17858daf74c2ea9cc93775172b567` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:36.541055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=48364ccf19434c4c9c09b4e5ce4bfc7b, role_id=adf17858daf74c2ea9cc93775172b567)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:36.544030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:36.564505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:36.565499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:36.568312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:36.568312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:36.568312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:36.568312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:36.569528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:36.604307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 309/616] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 24 14:12:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:36.615742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:36.645421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 308/617] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:12:36 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/48364ccf19434c4c9c09b4e5ce4bfc7b/roles/adf17858daf74c2ea9cc93775172b567 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:12:36.669412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:36.859357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8cc41-5cc1-4e53-a8a8-7c5ed9a62f95 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:36.904193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:36.904857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:36.905135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:36.905543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:36.924982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:36.926519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:36.944086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:36.945976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:36.947831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:36.947831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:36.947831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:36.947831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 310/618] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:36.952997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:36.971530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:37.066752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:37.138814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:37.207878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f816a-ba72-4e45-90a6-6b0d2d8816f7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:37.220804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988188-7afd-4b3a-a442-4e69b734e1f7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:37.288899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:37.289919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:37.290496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:37.290982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:37.292248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:37.292759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:37.295093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:37.295355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:37.296628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:37.297334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:37.298028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:37.333029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:37.344903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:37.346955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:37.348643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:37.351793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 309/619] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:37.362647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:37.367921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 311/620] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:37.384322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:37.639077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a13ba5-e150-495c-9f30-3854e619a327 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:37.664057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78601-cf69-4182-bb28-30e122a834a8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:37.679823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe55f8e-2322-4d36-aec7-1eb3c7f86359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:37.680460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe55f8e-2322-4d36-aec7-1eb3c7f86359 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:37.680738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe55f8e-2322-4d36-aec7-1eb3c7f86359 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:37.681017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe55f8e-2322-4d36-aec7-1eb3c7f86359 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:37.681706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe55f8e-2322-4d36-aec7-1eb3c7f86359 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:37.682046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe55f8e-2322-4d36-aec7-1eb3c7f86359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:37.684133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe55f8e-2322-4d36-aec7-1eb3c7f86359 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:37.690417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:37.694313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 312/621] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:37.695883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a12bc-ad00-49da-b360-5c1cc70f6876 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:37.696276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a12bc-ad00-49da-b360-5c1cc70f6876 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:37.696653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a12bc-ad00-49da-b360-5c1cc70f6876 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:37.703681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 313/622] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:37.704866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a12bc-ad00-49da-b360-5c1cc70f6876 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:37.705662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:37.740641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8320015-13af-435a-9cf0-be882c0cfecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:37.741660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320015-13af-435a-9cf0-be882c0cfecd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:37.742182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320015-13af-435a-9cf0-be882c0cfecd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:37.742647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320015-13af-435a-9cf0-be882c0cfecd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:37.756814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8320015-13af-435a-9cf0-be882c0cfecd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:37.761490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8320015-13af-435a-9cf0-be882c0cfecd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e8320015-13af-435a-9cf0-be882c0cfecd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:37.782610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:37.783875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:37.784750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:37.790262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 310/623] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:37.793485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcba31-3cee-4d1c-b2c5-e7e799213aed None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:37.793934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcba31-3cee-4d1c-b2c5-e7e799213aed None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:37.794496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcba31-3cee-4d1c-b2c5-e7e799213aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:37.832364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b8bcba31-3cee-4d1c-b2c5-e7e799213aed None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:37.834643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b8bcba31-3cee-4d1c-b2c5-e7e799213aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:37.837749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a12bc-ad00-49da-b360-5c1cc70f6876 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:37.881028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:37.882595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:37.883090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:37.883570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:37.897013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:37.899003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:37.933826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:37.935856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:37.937628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:37.937628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:37.937628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:37.937628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 314/624] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:37.938852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:37.991018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 311/625] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:37.992306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcba31-3cee-4d1c-b2c5-e7e799213aed None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:38.030632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:38.173925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f27e5-3e94-440b-8512-7f0721f930c9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:38.220051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcba31-3cee-4d1c-b2c5-e7e799213aed None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:38.236079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:38.253344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:38.254366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:38.254851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:38.255341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:38.256375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:38.256890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:38.258676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:38.289329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 315/626] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:38.296551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:38.318970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:38.320947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:38.321701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:38.323104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:38.326224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:38.327690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:38.329471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:38.356774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 312/627] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:38.362387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:38.492680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ca447-5735-48bf-bbe3-d7616af5add5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:38.497939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:38.511927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e635c0e-7304-47f1-9696-5f6933c1286f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:38.540535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:38.541489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:38.541871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:38.543549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/36c4eeb768c04233bcbfc786b3ba20b3/roles/adf17858daf74c2ea9cc93775172b567` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:38.560613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:38.561280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:38.561559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:38.561830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:38.562543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=36c4eeb768c04233bcbfc786b3ba20b3, role_id=adf17858daf74c2ea9cc93775172b567)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:38.564240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:38.571689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:38.573868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:38.587586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:38.588658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 313/628] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:12:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:38.590551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:38.626805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:38.641426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 316/629] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:12:38 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/36c4eeb768c04233bcbfc786b3ba20b3/roles/adf17858daf74c2ea9cc93775172b567 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:12:38.642956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:38.784599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df15c16-b71a-4d07-8157-9692b25e41ee None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:38.865834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb23495-3bf1-4b68-a259-8da79193d668 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:38.931207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:38.932226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:38.933095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:38.933637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:38.944060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:38.945037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:38.945476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:38.945900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:38.951408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:38.951908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:38.956544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:38.974191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:38.988408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:38.989431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:38.990326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:38.999665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 314/630] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:39.013029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:39.019612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 317/631] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:12:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:39.030514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:39.070790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:39.155593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:39.187825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba9de1-a422-4176-adfe-e40f40134b32 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:39.238263 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a43d20-a784-4e49-9290-7338c77e5329 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:39.263581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618d9a1-f365-44f5-b0ce-235fa546fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:39.264650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618d9a1-f365-44f5-b0ce-235fa546fb73 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.265139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618d9a1-f365-44f5-b0ce-235fa546fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.265622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618d9a1-f365-44f5-b0ce-235fa546fb73 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.266899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8618d9a1-f365-44f5-b0ce-235fa546fb73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:39.267434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8618d9a1-f365-44f5-b0ce-235fa546fb73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:39.270117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8618d9a1-f365-44f5-b0ce-235fa546fb73 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:39.280471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c3d45-6568-429f-ad08-971832e62a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:39.281190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c3d45-6568-429f-ad08-971832e62a86 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.281471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c3d45-6568-429f-ad08-971832e62a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.281744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c3d45-6568-429f-ad08-971832e62a86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.288901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 315/632] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:39.291406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e778eae-05ea-4f5b-a01e-fd47f9d4e43e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.292485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e778eae-05ea-4f5b-a01e-fd47f9d4e43e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.293764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e778eae-05ea-4f5b-a01e-fd47f9d4e43e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.297202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 316/633] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:39.298756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e778eae-05ea-4f5b-a01e-fd47f9d4e43e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:39.395751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c3d45-6568-429f-ad08-971832e62a86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:39.397309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c3d45-6568-429f-ad08-971832e62a86 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:39.510375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e778eae-05ea-4f5b-a01e-fd47f9d4e43e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:39.533734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 318/634] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:39.535138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f57eb6-fa58-4d89-b578-39e27d3678e2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.535423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f57eb6-fa58-4d89-b578-39e27d3678e2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.535697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f57eb6-fa58-4d89-b578-39e27d3678e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.545130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:39.570025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-33f57eb6-fa58-4d89-b578-39e27d3678e2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:39.570025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-33f57eb6-fa58-4d89-b578-39e27d3678e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:39.591308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:39.592452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.592954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.593498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.612593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:39.614393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:39.630843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:39.649718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:39.652187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:39.653973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:39.653973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:39.653973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:39.662873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 317/635] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:39.664789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:39.673442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 319/636] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:39.674466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f57eb6-fa58-4d89-b578-39e27d3678e2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:39.730763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:39.804134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f57eb6-fa58-4d89-b578-39e27d3678e2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:39.839378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9c2e-a74d-47a8-9ad0-2fbfcdd81a85 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:39.870729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c595c3-3e88-4049-ae09-68cc57ee503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:39.871698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c595c3-3e88-4049-ae09-68cc57ee503a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.872179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c595c3-3e88-4049-ae09-68cc57ee503a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.872643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c595c3-3e88-4049-ae09-68cc57ee503a None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.873978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c595c3-3e88-4049-ae09-68cc57ee503a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:39.874624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c595c3-3e88-4049-ae09-68cc57ee503a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:39.877613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c595c3-3e88-4049-ae09-68cc57ee503a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:39.880866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712c375-4980-4f73-a67e-905a41ab6be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:39.881832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712c375-4980-4f73-a67e-905a41ab6be4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.882449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712c375-4980-4f73-a67e-905a41ab6be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.882937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712c375-4980-4f73-a67e-905a41ab6be4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.896090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712c375-4980-4f73-a67e-905a41ab6be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:39.898036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712c375-4980-4f73-a67e-905a41ab6be4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:39.906077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 320/637] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:39.906077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f0f8a-7c83-4401-8d7f-9c4571251b54 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.906459 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f0f8a-7c83-4401-8d7f-9c4571251b54 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.906903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f0f8a-7c83-4401-8d7f-9c4571251b54 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.910146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 321/638] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:39.914184 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f0f8a-7c83-4401-8d7f-9c4571251b54 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f712c375-4980-4f73-a67e-905a41ab6be4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:39.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:39.938282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:39.939420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:39.939420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:39.939420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:39.946573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 318/639] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:39.949235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa961d-5655-47e0-9605-18c2a454c285 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:39.949710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa961d-5655-47e0-9605-18c2a454c285 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:39.950278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa961d-5655-47e0-9605-18c2a454c285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:39.982695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-18fa961d-5655-47e0-9605-18c2a454c285 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:39.984740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-18fa961d-5655-47e0-9605-18c2a454c285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:40.054315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:40.079514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 319/640] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:40.080983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa961d-5655-47e0-9605-18c2a454c285 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:40.132697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f0f8a-7c83-4401-8d7f-9c4571251b54 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:40.135097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:40.231592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa961d-5655-47e0-9605-18c2a454c285 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:40.236587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:40.238317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:40.239313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:40.240102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:40.263599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:40.267039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:40.282766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:40.284989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 322/641] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:40.286982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:40.305594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:40.307205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:40.308578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:40.309341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:40.330607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:40.341458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:40.343216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:40.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:40.349807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 320/642] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:40.351622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:40.430866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37ac1c-4011-4603-98b3-34495b1dd382 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:40.446884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:40.471046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:40.471670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:40.472006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:40.472322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:40.472946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:40.473243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:40.474707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:40.493771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 323/643] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:40.495785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:40.524522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:40.567290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e56f5-8133-43ad-87ab-80620d3b1b41 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:40.617330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a47fd-3510-4b0f-b8bd-52b8b776b2c2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:40.637029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:40.638146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:40.638672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:40.639161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:40.640619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:40.641157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:40.643998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:40.664117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:40.664745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:40.665125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:40.665307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:40.665948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:40.666300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:40.668343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:40.678585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 324/644] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:40.684377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:40.688792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 321/645] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:40.690430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:40.802711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:40.828150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e48bd-5e68-415c-bc97-0c1eee2f4343 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:40.864695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:40.869116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb594992-1b18-4b44-9c22-50d9ec7358ac None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:40.880352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:40.881498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:40.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:40.882484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:40.896288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:40.899098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:40.916542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:40.920010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:40.923601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 325/646] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:40.925476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:40.944037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:40.945463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:40.946103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:40.946584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:40.968807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:40.978464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:40.980892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:40.982602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 322/647] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:40.983978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:41.115983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a21675-a149-441e-a7c2-c41abbae359c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:41.159981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38e710-bf4c-4819-998e-7588827c35b0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:41.171753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:41.172504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.172805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.172944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.185655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:41.187297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:41.191499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:41.208705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:41.210590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:41.212383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:41.212383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:41.212383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:41.213180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 326/648] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:41.214764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:41.230411 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:41.231474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.231990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.232473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.286025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:41.347839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:41.349517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:41.357060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cf0ae-1445-4cee-9337-a21112f11b7c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:41.412891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:41.413602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.413948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.414313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.415097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:41.415456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:41.417045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:41.433931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 327/649] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:41.435062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:41.507464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 323/650] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:41.508275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:41.525557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:41.577874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833bfda-2c97-46dd-934f-dfbec95b88a0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:41.584788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:41.616062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787905b5-fde7-4bc8-8e10-210463f8177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:41.616680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787905b5-fde7-4bc8-8e10-210463f8177f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.616957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787905b5-fde7-4bc8-8e10-210463f8177f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.617228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787905b5-fde7-4bc8-8e10-210463f8177f None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.617891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787905b5-fde7-4bc8-8e10-210463f8177f None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:41.618233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787905b5-fde7-4bc8-8e10-210463f8177f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:41.619627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787905b5-fde7-4bc8-8e10-210463f8177f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:41.629272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 328/651] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:41.630606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6eebf-2fe3-4e56-87de-362aac1f3b46 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.630890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6eebf-2fe3-4e56-87de-362aac1f3b46 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.631164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6eebf-2fe3-4e56-87de-362aac1f3b46 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.632482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 329/652] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:41.637684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6eebf-2fe3-4e56-87de-362aac1f3b46 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:41.638023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e09fd-cc21-4484-ac1e-f03f82cc27f4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:41.691705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:41.692665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.693106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.693546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.694677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:41.695488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:41.697692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:41.723997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 324/653] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:41.723997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:41.780049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6eebf-2fe3-4e56-87de-362aac1f3b46 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:41.805338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:41.834653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:41.835615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.835997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.836315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.837667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=613e985e94c54913a5e6b93367157fc0) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:41.838307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:41.841361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:41.852270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 330/654] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:41 2022] GET /identity/v3/role_assignments?user.id=613e985e94c54913a5e6b93367157fc0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:41.853661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:41.879474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282221cf-b489-4787-864e-9b8c9042aaa8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:41.916464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:41.937784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cff616-2678-4721-8264-5e9053ad14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:41.938489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cff616-2678-4721-8264-5e9053ad14a1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.938769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cff616-2678-4721-8264-5e9053ad14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.939086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cff616-2678-4721-8264-5e9053ad14a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:41.952569 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cff616-2678-4721-8264-5e9053ad14a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:41.954366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cff616-2678-4721-8264-5e9053ad14a1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-50cff616-2678-4721-8264-5e9053ad14a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:41.981394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:41.983489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:41.985709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:41.985709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:41.985709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:41.985709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 325/655] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:41.985709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186016-053e-4d1c-91da-019af32ce5f1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:41.986394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186016-053e-4d1c-91da-019af32ce5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:41.986394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186016-053e-4d1c-91da-019af32ce5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:42.017190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a3186016-053e-4d1c-91da-019af32ce5f1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:42.019144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a3186016-053e-4d1c-91da-019af32ce5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:42.084071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd77924-498c-4a96-8e6b-df9422c65a4c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:42.148799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:42.149911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:42.150405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:42.150855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:42.151593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 326/656] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:42.152886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3186016-053e-4d1c-91da-019af32ce5f1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:42.170392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:42.173823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:42.206231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:42.210543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:42.212554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:42.212554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:42.212554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:42.212554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 331/657] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:42.216678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:42.309050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:42.378719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3186016-053e-4d1c-91da-019af32ce5f1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:42.382733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:42.446827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa31706-3703-425a-bfc4-9be38bb703eb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:42.451839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:42.453366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:42.454185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:42.454946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:42.470125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:42.473755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:42.499933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:42.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:12:42.503716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:42.508275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 327/658] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:42.510279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:42.519583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:42.520621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:42.520903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:42.521318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:42.522559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:42.523087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:42.525896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:42.547447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 332/659] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:42.548705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:42.694260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e510-4ada-4413-bc16-3f650f707782 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:42.709085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:42.721959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101b80-52e2-4a82-8792-bf1fef6f888a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:42.758954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:42.759981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:42.760463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:42.761907 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:42.763585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:42.764123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:42.766958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:42.790699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 328/660] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:42.796054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:42.799240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:42.800602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:42.802265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:42.802571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:42.803391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:42.803694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:42.805154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:42.834081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 333/661] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 24 14:12:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:42.834081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:42.857347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:42.947643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478c976-65af-4382-b738-0f3483d4e5f8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:42.990483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:42.991123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:42.991395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:42.991663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.000308 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:43.001770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:43.004121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c1cf5-44ba-4b8d-bb68-d572fbefe9c3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:43.015506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:43.017069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 334/662] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:43.018760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:43.037009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:43.068598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:43.069333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:43.069609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:43.069876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.083973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:43.090147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:43.091162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:43.092093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 329/663] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:43.093386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:43.166064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801f0b6-3034-4c5e-93b4-017b12f5b288 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:43.223528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:43.224556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:43.225407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:43.225860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:43.226335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.241099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:43.243529 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:43.257176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfd503-908d-4552-8adc-20be43d2c6bb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:43.260057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:43.261633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 14:12:43.263061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:43.268035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 335/664] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:43.268035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:43.322451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:43.324379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:43.324860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:43.325325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.384165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:43.435497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98146b2e-603f-4db6-9a4b-3b259ce252cd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:43.455486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:43.457041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:43.504813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:43.505928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:43.506478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:43.507073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.508272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:43.508777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:43.511627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:43.538125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 336/665] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:43.539588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:43.585777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:43.614675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 330/666] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:43.615592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:43.713981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1444d3-c08c-48a7-ba5c-9c770662aaa6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:43.751663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:43.757484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:43.758048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:43.758383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:43.766546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.766546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:43.766546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:43.766546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:43.780374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 337/667] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:43.781815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:43.786557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565cea-0bed-4c36-bac5-af78207f69f4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:43.845405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:43.846452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:43.846894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:43.847349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.848572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:43.848953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:43.850905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:43.884758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 331/668] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:43.886928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:43.943060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b8147-d966-4e28-a6d0-1ea86d96a5c2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:43.970450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:43.987296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:43.987926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:43.988814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:43.988814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:43.989287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=aec389ead5164e778a9ee73e7ba7fc50) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:43.989594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:43.991160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:43.998901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 338/669] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/role_assignments?user.id=aec389ead5164e778a9ee73e7ba7fc50&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:43.999789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:44.100326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24c953-4fb6-457b-b4ab-74f37b790be9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:44.138413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:44.178438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:44.179594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:44.180641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:44.181102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:44.205622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:44.208696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:44.231047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ceb72-f9f3-4250-ade7-3db10120e916 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:44.243670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:44.246321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:44.248399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:44.248399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:44.248399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:44.253289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 332/670] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:44.254852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:44.311119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:44.311882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:44.312246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:44.313504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:44.335208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:44.337299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:44.364269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:44.366051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:44.369641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:44.369641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:44.369641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:44.369641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 339/671] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:44.369641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:44.420983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:44.435464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28561d-faa2-4ffe-843a-95445a8beb7e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:44.511169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:44.512526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:44.513081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:44.513561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:44.521279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb16163-d487-4cb2-9bbd-34067bf56342 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:44.533831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:44.563112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275bce8-ad8e-4b1d-980e-806f971a7fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:44.563795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275bce8-ad8e-4b1d-980e-806f971a7fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:44.564179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275bce8-ad8e-4b1d-980e-806f971a7fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:44.564457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275bce8-ad8e-4b1d-980e-806f971a7fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:44.565123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275bce8-ad8e-4b1d-980e-806f971a7fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:44.565424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275bce8-ad8e-4b1d-980e-806f971a7fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:44.566848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275bce8-ad8e-4b1d-980e-806f971a7fa1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:44.595564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 340/672] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:44.595564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c9701-bc5c-4206-adb3-253bd6e34b6f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:44.595564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c9701-bc5c-4206-adb3-253bd6e34b6f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:44.595564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c9701-bc5c-4206-adb3-253bd6e34b6f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:44.598650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 341/673] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:44.598838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c9701-bc5c-4206-adb3-253bd6e34b6f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:44.631908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:44.633379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:44.741984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c9701-bc5c-4206-adb3-253bd6e34b6f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:44.765439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:44.788281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 333/674] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:44.788281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:44.806924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47c1d8-d15e-47a9-bc90-c6d33f8e85f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:44.808210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47c1d8-d15e-47a9-bc90-c6d33f8e85f8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:44.808550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47c1d8-d15e-47a9-bc90-c6d33f8e85f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:44.808886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47c1d8-d15e-47a9-bc90-c6d33f8e85f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:44.809807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47c1d8-d15e-47a9-bc90-c6d33f8e85f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:44.810288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47c1d8-d15e-47a9-bc90-c6d33f8e85f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:44.811824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47c1d8-d15e-47a9-bc90-c6d33f8e85f8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:44.832463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 342/675] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:44.832463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826be7f-9f70-49ff-98f1-b388daf80262 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:44.832463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826be7f-9f70-49ff-98f1-b388daf80262 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:44.832463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826be7f-9f70-49ff-98f1-b388daf80262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:44.852472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6826be7f-9f70-49ff-98f1-b388daf80262 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:44.854178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6826be7f-9f70-49ff-98f1-b388daf80262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:44.884635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:44.931657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a46af-9464-4529-bf62-65e2cc7be619 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:44.939832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 343/676] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:44.940969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826be7f-9f70-49ff-98f1-b388daf80262 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.017125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:45.018536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:45.019661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:45.021117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:45.041774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:45.046111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:45.070170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:45.071835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:45.074281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:45.076581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 334/677] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:45.077416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.092368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826be7f-9f70-49ff-98f1-b388daf80262 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:45.157157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:45.172586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:45.174639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:45.174870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:45.175528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:45.203529 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:45.207991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:45.221175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfb224-3dc6-4f6b-96cc-f180c96eb6ce None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:45.249026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:45.251570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:45.253416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:45.253416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:45.253416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:45.257171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 344/678] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:45.259027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.275314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:45.276191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:45.276572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:45.276963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:45.287798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:45.288916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:45.297275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:45.306316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:45.314774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 335/679] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:45.316688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.457072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0729643-6197-4e36-8036-388ee219c7f6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:45.472019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d67783-cc6d-4f10-8381-506ad64c0815 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:45.473877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:45.500876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:45.501633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:45.501977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:45.502281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:45.502985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:45.503967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:45.505839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:45.520829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:45.521430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:45.521696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:45.521968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:45.522706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:45.523004 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:45.530503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 336/680] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:45.531132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:45.531291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.546248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 345/681] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:45.547137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.656045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9ff1d-d670-402f-ab37-008107a81c5d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:45.685600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:45.703393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:45.704330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:45.704722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:45.705175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:45.706374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:45.706911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:45.709630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:45.743164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02140c7f-e67c-41a5-a854-37cf8c88c97e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:45.750065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 337/682] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 24 14:12:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:45.750065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.792182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:45.793016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:45.793473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:45.793925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:45.806920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:45.811541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:45.819686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:45.821007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 346/683] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:45.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:45.947421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf5946-6372-46e8-ae80-88ce0dc7c1f3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:46.006728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498366ba-f6d1-43ff-9d68-cd1f94a2d5ea None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:46.008885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:46.009818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.010299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.011270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.023227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:46.025606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:46.045976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:46.056184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 338/684] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:46.058446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:46.077832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:46.085164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:46.086208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.092229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.092229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.092229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=9daf5be8a3dc48e78294882c62b8e60f) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:46.092229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:46.092229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:46.109432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 347/685] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:45 2022] GET /identity/v3/role_assignments?user.id=9daf5be8a3dc48e78294882c62b8e60f&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:46.109432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:46.270579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4735ab-e830-47ba-b105-86236ac01071 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:46.285486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:46.309956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0dad0-4c7a-41e9-9112-3fa5d885304a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:46.341646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:46.342967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.344616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.344616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.345584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:46.346595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:46.348697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:46.368145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 348/686] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:46.370605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:46.391257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:46.392334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.392824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.393297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.485249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:46.518823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:46.521402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:46.568404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067fa58-14fe-40f5-8d74-8e2de0d96e60 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:46.637982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1059c-7a67-4882-8873-81eb711e6b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:46.638781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1059c-7a67-4882-8873-81eb711e6b58 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.639152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1059c-7a67-4882-8873-81eb711e6b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.639495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1059c-7a67-4882-8873-81eb711e6b58 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.640294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1059c-7a67-4882-8873-81eb711e6b58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:46.640662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1059c-7a67-4882-8873-81eb711e6b58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:46.642271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1059c-7a67-4882-8873-81eb711e6b58 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:46.662447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 349/687] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:46.663866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270144-af30-4145-a611-d862255aac70 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.664220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270144-af30-4145-a611-d862255aac70 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.664567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270144-af30-4145-a611-d862255aac70 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.671920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 350/688] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:46.671920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270144-af30-4145-a611-d862255aac70 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:46.687534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:46.702096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 339/689] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:46.703230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:46.860310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270144-af30-4145-a611-d862255aac70 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:46.860977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f7d09-69d6-43b2-9901-9fce3f7b18b2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:46.883670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:46.906422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:46.907078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.907351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.907954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.910532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537324ea-8085-4d06-b83b-8e1d35414c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:46.911162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537324ea-8085-4d06-b83b-8e1d35414c9a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.911442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537324ea-8085-4d06-b83b-8e1d35414c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.911724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537324ea-8085-4d06-b83b-8e1d35414c9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.912537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537324ea-8085-4d06-b83b-8e1d35414c9a None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=48364ccf19434c4c9c09b4e5ce4bfc7b) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:46.912920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537324ea-8085-4d06-b83b-8e1d35414c9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:46.914482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537324ea-8085-4d06-b83b-8e1d35414c9a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:46.921643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:46.930268 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 340/690] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:46 2022] GET /identity/v3/role_assignments?user.id=48364ccf19434c4c9c09b4e5ce4bfc7b&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:46.930552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:46.930697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a437-7710-442e-ae3b-0848a016a6f3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:46.930697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a437-7710-442e-ae3b-0848a016a6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:46.930697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a437-7710-442e-ae3b-0848a016a6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:46.949344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:46.950984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:46.952727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:46.952727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:46.952727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:46.959765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 351/691] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:46.959765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:47.098671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1aa1a437-7710-442e-ae3b-0848a016a6f3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:47.100521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1aa1a437-7710-442e-ae3b-0848a016a6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:47.140505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b58f6-e4b8-4bb9-8f16-6e0d52578efd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:47.186122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:47.186914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:47.187266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:47.187613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:47.188368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:47.202551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:47.204022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:47.222742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:47.223842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:47.224716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:47.224716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:47.224716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:47.224896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 341/692] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:47.229095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 352/693] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:47.230951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:47.234766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a437-7710-442e-ae3b-0848a016a6f3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:47.263859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:47.383674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c274a2b-53e3-41f5-afc5-93299eb275b2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:47.439055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a437-7710-442e-ae3b-0848a016a6f3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:47.454838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:47.455554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:47.455844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:47.456114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:47.456790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:47.457095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:47.458537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:47.508389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 353/694] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:47.515214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:47.521399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:47.522580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:47.523072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:47.523565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:47.525380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:47.526115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:47.529547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:47.564437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 342/695] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:47.578208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:47.687371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:47.691837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:47.715453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54530c44-992c-4fa6-a51b-fadc31eeae78 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:47.753910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed420b0-03c8-42bf-a3de-ca6cf557fd6d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:47.755971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:47.756627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:47.756901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:47.757172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:47.767125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:47.768631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:47.782103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:47.783359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 354/696] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:47.789348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:47.806601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:47.808922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:47.809484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:47.810122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:47.824392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:47.827104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:47.844153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:47.845184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:47.846813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:47.850028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 343/697] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:12:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:47.851110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:47.925132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663b9f0-faa6-47cd-9672-ee3ac6f43022 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:47.968215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:47.968841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:47.969118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:47.969386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:47.983893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:47.985202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9b71c-66ea-4168-8d8c-361afae88bdf None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:47.985430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:47.987607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:47.989295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:48.005368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:48.006664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:48.008015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:48.008015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:48.008015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:48.011468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 355/698] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:48.012517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:48.031585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:48.033069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.033591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.034209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.035176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:48.035710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:48.037533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:48.057023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 344/699] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:48.059485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:48.134635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffcba6-3983-4085-9e8f-8d402eea5554 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:48.174248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:48.175017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.175295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.175559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.176669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:48.176805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:48.178477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:48.180269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5056b96-6e2d-4184-b9b1-71b14a750274 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:48.205169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 356/700] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:12:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:48.206268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:48.222898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:48.224144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.224637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.225216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.226161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:48.226763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:48.228630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:48.243957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 345/701] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:48.245634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:48.278105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:48.281195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:48.342783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20805f-bb1f-4705-b8e9-b108666764eb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:48.368561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8f9f7-0991-4eb7-a564-b41b4636b0eb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:48.393264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:48.393908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.394161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.394409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.395136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=36c4eeb768c04233bcbfc786b3ba20b3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:48.395440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:48.397056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:48.406173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 357/702] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:48 2022] GET /identity/v3/role_assignments?user.id=36c4eeb768c04233bcbfc786b3ba20b3&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:48.407438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:48.420127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5f05b-6c9c-4073-bbda-21e2ebef0b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:48.421519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5f05b-6c9c-4073-bbda-21e2ebef0b84 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.423646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5f05b-6c9c-4073-bbda-21e2ebef0b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.424367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5f05b-6c9c-4073-bbda-21e2ebef0b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.546785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5f05b-6c9c-4073-bbda-21e2ebef0b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:48.549694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5f05b-6c9c-4073-bbda-21e2ebef0b84 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:48.585957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f339403-fef7-4a93-b80b-3fe6745862ba None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:48.595974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:48.616727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:48.650550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d639b31-76dd-41e0-8e5b-1d8d10528cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:48.651891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d639b31-76dd-41e0-8e5b-1d8d10528cf0 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.652259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d639b31-76dd-41e0-8e5b-1d8d10528cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.652419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d639b31-76dd-41e0-8e5b-1d8d10528cf0 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.653261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d639b31-76dd-41e0-8e5b-1d8d10528cf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:48.654942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d639b31-76dd-41e0-8e5b-1d8d10528cf0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:48.728332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 346/703] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:48.730115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49acd-6803-466a-83ca-cfd464aae839 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.730479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49acd-6803-466a-83ca-cfd464aae839 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.730756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49acd-6803-466a-83ca-cfd464aae839 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.732299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 347/704] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:48.733854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49acd-6803-466a-83ca-cfd464aae839 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:48.774676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 358/705] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:12:48 2022] POST /identity/v3/users => generated 254 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:48.776694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6b670-a8bc-4777-bfe7-a56370bab297 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:48.777161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6b670-a8bc-4777-bfe7-a56370bab297 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:48.778056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6b670-a8bc-4777-bfe7-a56370bab297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:48.803664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b4f6b670-a8bc-4777-bfe7-a56370bab297 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:48.805644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b4f6b670-a8bc-4777-bfe7-a56370bab297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:48.912828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49acd-6803-466a-83ca-cfd464aae839 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:48.941113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 359/706] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:49.001304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:49.012968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:49.014139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:49.014648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:49.016968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:49.037732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:49.040584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:49.074214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:49.075216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:49.076126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:49.076126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:49.076126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:49.076747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 348/707] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:49.084965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6b670-a8bc-4777-bfe7-a56370bab297 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:49.293067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:49.312129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6b670-a8bc-4777-bfe7-a56370bab297 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:49.314131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:49.454585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:49.455413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:49.455614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:49.456043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:49.457020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:49.457707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:49.460320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:49.464818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb5c39-b229-4385-be26-aafaec13580c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:49.504171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 360/708] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:49.508204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:49.508868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:49.509147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:49.509420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:49.510051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:49.512665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:49.517345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:49.525512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:49.528452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:49.530719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 349/709] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 24 14:12:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:49.535869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:49.646142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:49.740101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407964c-c82f-45bc-a22f-65d6643a50c5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:49.762169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9306b1-1128-46ad-933f-94c44ca1c3b1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:49.798934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17afb17-f7c2-4e44-9c74-dca9c4798021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:49.800660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17afb17-f7c2-4e44-9c74-dca9c4798021 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:49.801577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17afb17-f7c2-4e44-9c74-dca9c4798021 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:49.802355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17afb17-f7c2-4e44-9c74-dca9c4798021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:49.809911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb0a26-2fde-4fc7-ac8b-c918a014acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:49.810574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb0a26-2fde-4fc7-ac8b-c918a014acec None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:49.810849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb0a26-2fde-4fc7-ac8b-c918a014acec None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:49.811118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb0a26-2fde-4fc7-ac8b-c918a014acec None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:49.811762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb0a26-2fde-4fc7-ac8b-c918a014acec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:49.812061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb0a26-2fde-4fc7-ac8b-c918a014acec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:49.813476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb0a26-2fde-4fc7-ac8b-c918a014acec None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:49.817926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17afb17-f7c2-4e44-9c74-dca9c4798021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:49.819210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 350/710] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 24 14:12:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:49.820541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa5406-c35d-4461-bde6-07cad45e52f0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:49.820846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa5406-c35d-4461-bde6-07cad45e52f0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:49.821093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa5406-c35d-4461-bde6-07cad45e52f0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:49.822364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17afb17-f7c2-4e44-9c74-dca9c4798021 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:49.829539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 351/711] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:49.829539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa5406-c35d-4461-bde6-07cad45e52f0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a17afb17-f7c2-4e44-9c74-dca9c4798021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:49.856425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:49.858534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:49.860221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:49.869042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 361/712] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:49.872630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f52b9-0766-493f-a1b4-3445ceb41e93 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:49.872630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f52b9-0766-493f-a1b4-3445ceb41e93 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:49.872630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f52b9-0766-493f-a1b4-3445ceb41e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:49.906078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-083f52b9-0766-493f-a1b4-3445ceb41e93 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:49.909136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-083f52b9-0766-493f-a1b4-3445ceb41e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:49.943288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:49.955292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa5406-c35d-4461-bde6-07cad45e52f0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:49.960441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:50.003193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:50.003768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:50.004044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:50.004313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:50.005035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:50.005340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:50.006944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:50.014604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 352/713] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 24 14:12:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:50.015644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:50.061022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 362/714] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:50.218413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f463e-7718-46a7-9c2c-6bcbf2db3663 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:50.269831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:50.270522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:50.270807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:50.271078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:50.271810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:50.272117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:50.273758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:50.289611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 353/715] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:50.296211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f52b9-0766-493f-a1b4-3445ceb41e93 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:50.449281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:50.514721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:50.556042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f52b9-0766-493f-a1b4-3445ceb41e93 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:50.635689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:50.637806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:50.638304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:50.638751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:50.640128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=613e985e94c54913a5e6b93367157fc0) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:50.640667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:50.654189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:50.685120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 363/716] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:50 2022] GET /identity/v3/role_assignments?user.id=613e985e94c54913a5e6b93367157fc0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:50.769328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b68b7b-906d-4efa-8dad-731847d0b847 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:50.864684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:50.865758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:50.866438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:50.866930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:50.867762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:50.869411 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:50.880919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:50.881972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:12:50.882847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:50.885880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 354/717] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:50 2022] GET /identity/v3/services/network => generated 88 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:50.895115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:50.955828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:51.177230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29521650-618d-4d1e-afbf-904b7de5dd5d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:51.234573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426a61a-8068-414e-9f1a-b71700bb0556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:51.235604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a61a-8068-414e-9f1a-b71700bb0556 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:51.236088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a61a-8068-414e-9f1a-b71700bb0556 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:51.236583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a61a-8068-414e-9f1a-b71700bb0556 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:51.237448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a61a-8068-414e-9f1a-b71700bb0556 None admin] RBAC: Adding query filter params (name=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:51.237958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a61a-8068-414e-9f1a-b71700bb0556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:51.239689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a61a-8068-414e-9f1a-b71700bb0556 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:51.246430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 364/718] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:12:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:51.254664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:51.404847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:51.427099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bf90a-1289-4e1a-b994-b25289f020e8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:51.520862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb59c31-59aa-4f71-931f-86707d105a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:51.521844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb59c31-59aa-4f71-931f-86707d105a39 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:51.522380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb59c31-59aa-4f71-931f-86707d105a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:51.522867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb59c31-59aa-4f71-931f-86707d105a39 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:51.524044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb59c31-59aa-4f71-931f-86707d105a39 None admin] RBAC: Adding query filter params (type=network) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:51.524595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb59c31-59aa-4f71-931f-86707d105a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:51.527471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb59c31-59aa-4f71-931f-86707d105a39 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:51.541234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 355/719] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:12:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:51.592557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae0ff-e905-4628-a3fc-85c76b060c7f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:51.593100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae0ff-e905-4628-a3fc-85c76b060c7f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:51.593100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae0ff-e905-4628-a3fc-85c76b060c7f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:51.594943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 365/720] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:51.601058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd946-4a7d-46a2-9e35-341a0e15075d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:51.601382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd946-4a7d-46a2-9e35-341a0e15075d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:51.601612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd946-4a7d-46a2-9e35-341a0e15075d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:51.623054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-202dd946-4a7d-46a2-9e35-341a0e15075d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:51.624571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-202dd946-4a7d-46a2-9e35-341a0e15075d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:51.710283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 356/721] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:51.924451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e24e6e-482b-49de-87ce-27bf72fa3bf8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:51.924863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e24e6e-482b-49de-87ce-27bf72fa3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:51.925047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e24e6e-482b-49de-87ce-27bf72fa3bf8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:51.926915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 366/722] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:51.933085 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a00a6-8d5c-4f11-a586-e037024e7bed None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:51.933598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a00a6-8d5c-4f11-a586-e037024e7bed None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:51.933598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a00a6-8d5c-4f11-a586-e037024e7bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:51.957712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f51a00a6-8d5c-4f11-a586-e037024e7bed None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:51.958998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f51a00a6-8d5c-4f11-a586-e037024e7bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:52.064602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 357/723] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:52.103679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e24e6e-482b-49de-87ce-27bf72fa3bf8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:52.223212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:52.248056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e24e6e-482b-49de-87ce-27bf72fa3bf8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:52.298822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:52.299485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:52.299760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:52.300034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:52.319316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:52.324676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:52.325705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:52.327047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 367/724] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:52.333044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a00a6-8d5c-4f11-a586-e037024e7bed None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:52.378392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:52.431568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:52.530700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a00a6-8d5c-4f11-a586-e037024e7bed None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:52.576676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:52.577307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:52.577589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:52.577863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:52.578580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:52.578886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:52.580313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:52.592119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 358/725] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:52.599055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:52.704783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25655f-94d7-462d-884e-635e6ed62a94 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:52.772829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:52.773909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:52.774433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:52.774905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:52.789547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:52.802315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:52.807977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:52.809281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:52.810156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 368/726] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:12:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:52.813244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:52.823867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:52.847477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e035827-88ab-4f20-9ac1-63056a185206 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:52.900069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:52.900998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:52.901885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:52.901885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:52.929232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:52.930893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:52.957496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:52.960286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:52.962789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:52.962789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:52.962789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:52.962789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 359/727] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:52.971529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:53.071827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001d79-77b7-401d-834f-59a70ee668fb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:53.129935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27933-6880-44da-9a33-511177ef8313 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:53.167568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:53.174898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:53.175893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.183021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.183021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.183021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:53.183021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:53.183021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:53.188433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e6ba2-1088-4b13-a857-dfda617d8308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:53.189447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e6ba2-1088-4b13-a857-dfda617d8308 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.189963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e6ba2-1088-4b13-a857-dfda617d8308 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.190500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e6ba2-1088-4b13-a857-dfda617d8308 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.191409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e6ba2-1088-4b13-a857-dfda617d8308 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:53.194109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e6ba2-1088-4b13-a857-dfda617d8308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:53.195906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e6ba2-1088-4b13-a857-dfda617d8308 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:53.203963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 360/728] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:53.210069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:53.217156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 369/729] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:12:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:53.225182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca57d6-ad75-447d-a2f0-65ce688a11f1 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.225668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca57d6-ad75-447d-a2f0-65ce688a11f1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.226207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca57d6-ad75-447d-a2f0-65ce688a11f1 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.229949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 370/730] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:53.238751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca57d6-ad75-447d-a2f0-65ce688a11f1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:53.352366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789fe87-5974-4845-852d-c4329623a42b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:53.387738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:53.439565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8dbc8-72ce-4c56-9b71-8a7735b329a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:53.440531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca57d6-ad75-447d-a2f0-65ce688a11f1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:53.441448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8dbc8-72ce-4c56-9b71-8a7735b329a7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.442305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8dbc8-72ce-4c56-9b71-8a7735b329a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.442594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8dbc8-72ce-4c56-9b71-8a7735b329a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.454408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8dbc8-72ce-4c56-9b71-8a7735b329a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:53.455895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8dbc8-72ce-4c56-9b71-8a7735b329a7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-05a8dbc8-72ce-4c56-9b71-8a7735b329a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:53.469834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:53.471785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 361/731] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849066e-18c4-46a6-9c16-26c23b058e56 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849066e-18c4-46a6-9c16-26c23b058e56 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.475202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849066e-18c4-46a6-9c16-26c23b058e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.495221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3849066e-18c4-46a6-9c16-26c23b058e56 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:53.496467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3849066e-18c4-46a6-9c16-26c23b058e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:53.538421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:53.539075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.539406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.539657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.556915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:53.557880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:53.575175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:53.576969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:53.578768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:53.578768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:53.578768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:53.578768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 371/732] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:53.578768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:53.585324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:53.615596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 362/733] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:53.616527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849066e-18c4-46a6-9c16-26c23b058e56 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:53.727383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf63a4-c6a2-436d-864e-f7bc3eeb644c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:53.747382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:53.775447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dced9-e44b-47f3-8ca2-981f319b89a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:53.776969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dced9-e44b-47f3-8ca2-981f319b89a4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.777346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dced9-e44b-47f3-8ca2-981f319b89a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.777720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dced9-e44b-47f3-8ca2-981f319b89a4 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.778621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dced9-e44b-47f3-8ca2-981f319b89a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:53.779616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dced9-e44b-47f3-8ca2-981f319b89a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:53.781276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dced9-e44b-47f3-8ca2-981f319b89a4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:53.810241 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 372/734] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 24 14:12:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:53.815643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea33ef2-10bf-438e-80da-c807159fdd04 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.816424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea33ef2-10bf-438e-80da-c807159fdd04 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.817157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea33ef2-10bf-438e-80da-c807159fdd04 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.818617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849066e-18c4-46a6-9c16-26c23b058e56 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:53.820584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 373/735] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:53.822468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea33ef2-10bf-438e-80da-c807159fdd04 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:53.873123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5ea8a-f676-4758-959d-6decf779779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:53.873839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ea8a-f676-4758-959d-6decf779779f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.874239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ea8a-f676-4758-959d-6decf779779f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.874625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5ea8a-f676-4758-959d-6decf779779f None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.875638 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ea8a-f676-4758-959d-6decf779779f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:53.876047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ea8a-f676-4758-959d-6decf779779f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:53.878195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5ea8a-f676-4758-959d-6decf779779f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:53.894433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 363/736] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:53.895946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fd186-7a36-4b41-8a0d-543c9d607213 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:53.896350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fd186-7a36-4b41-8a0d-543c9d607213 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:53.896707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fd186-7a36-4b41-8a0d-543c9d607213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:53.919675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-880fd186-7a36-4b41-8a0d-543c9d607213 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:53.921396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-880fd186-7a36-4b41-8a0d-543c9d607213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:53.967377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:54.036016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 364/737] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:54.038054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fd186-7a36-4b41-8a0d-543c9d607213 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:54.042449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea33ef2-10bf-438e-80da-c807159fdd04 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:54.134999 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb3579-e031-4a54-aa63-3febe90e7706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:54.136811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb3579-e031-4a54-aa63-3febe90e7706 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.137387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb3579-e031-4a54-aa63-3febe90e7706 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.137958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb3579-e031-4a54-aa63-3febe90e7706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.236668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:54.240340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fd186-7a36-4b41-8a0d-543c9d607213 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:54.287249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb3579-e031-4a54-aa63-3febe90e7706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:54.287492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:54.287907 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.288182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.288474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.292189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb3579-e031-4a54-aa63-3febe90e7706 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:54.298172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:54.299774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:54.313522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:54.315432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:12:54.317283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:54.319708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 365/738] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:12:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:54.320708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:54.358055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:54.445795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 374/739] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:54.447166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0352d-491b-4eb6-92f3-f5834fc6cc29 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.448100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0352d-491b-4eb6-92f3-f5834fc6cc29 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.448412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0352d-491b-4eb6-92f3-f5834fc6cc29 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.449805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 375/740] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:54.450703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0352d-491b-4eb6-92f3-f5834fc6cc29 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:54.453627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d5b9c-4e2c-4ba5-aad3-74a3ec62693c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:54.500474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:54.501267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.501631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.502031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.520496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:12:54.525476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:54.526759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:54.527869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 366/741] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:12:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:54.528669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:54.558300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:54.609375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0352d-491b-4eb6-92f3-f5834fc6cc29 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:54.671114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:54.671888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.672260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.672670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.673020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:54.673716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:54.674254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:54.676220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:54.686458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eb0b2-0e70-4dbf-853f-8a3f715a4c7c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:54.700980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 376/742] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 24 14:12:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:54.702528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:54.748959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5aea75-f1eb-4d16-8bc0-92cb5c4b5616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:54.749920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5aea75-f1eb-4d16-8bc0-92cb5c4b5616 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.750402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5aea75-f1eb-4d16-8bc0-92cb5c4b5616 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.750844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5aea75-f1eb-4d16-8bc0-92cb5c4b5616 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.751925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5aea75-f1eb-4d16-8bc0-92cb5c4b5616 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:54.754240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5aea75-f1eb-4d16-8bc0-92cb5c4b5616 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:54.839054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84c258-6ee7-4262-abe1-0e2f6cd0ed42 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:54.874088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:54.880767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:54.881646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.881922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.882230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.894169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:54.896077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:54.903632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 367/743] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:12:54 2022] POST /identity/v3/services => generated 251 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:54.905141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593394bd-8d86-4f7f-a17f-bfc3ab774544 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:54.905428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593394bd-8d86-4f7f-a17f-bfc3ab774544 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:54.908521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593394bd-8d86-4f7f-a17f-bfc3ab774544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:54.925026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:54.926924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:54.928629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:54.928629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:54.928629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:54.928629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 377/744] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:54.928629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:54.936827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-593394bd-8d86-4f7f-a17f-bfc3ab774544 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:54.938267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-593394bd-8d86-4f7f-a17f-bfc3ab774544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:55.058134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 368/745] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 14:12:55.058134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593394bd-8d86-4f7f-a17f-bfc3ab774544 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:55.117411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e38c-0606-4221-805b-ecfe600f03b6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:55.180086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:55.180727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:55.181008 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:55.181289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:55.182673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:55.182985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:55.184408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:55.197805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 378/746] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:12:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:55.199852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:55.202762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:55.251706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593394bd-8d86-4f7f-a17f-bfc3ab774544 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:55.321855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:55.325256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:55.325256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:55.325256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:55.352885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:55.476985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fe4a9-5b46-472f-8b1f-d4037fd489bd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:55.485977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:55.489564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:55.532002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:55.533671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:55.535274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:55.536001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:55.537438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:55.538228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:55.540995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:55.569613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 379/747] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:55.569613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:55.610522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:55.651485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 369/748] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:12:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 14:12:55.652885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:55.773819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a1a7-4cdf-4a5d-b138-7d6515b46fa7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:55.780139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:55.805670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf54c2-a98a-4dfd-bd0b-c6c89f714076 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:55.843512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:55.844592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:55.845123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:55.845619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:55.853098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:55.854479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:55.854888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:55.855236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:55.855912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:55.857351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:55.867324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:55.870195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:55.900915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:55.902879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:55.904577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:55.904577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:55.904577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:55.904577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 380/749] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:55.905148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:55.962078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 370/750] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:12:55 2022] POST /identity/v3/services => generated 240 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:12:55.962078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:56.014366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:56.112982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd6ca9-d0a9-417e-967b-b7f86af816cc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.139200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3bf70-5269-4df4-851e-61ea02258c05 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.161185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:56.162555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:56.162846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:56.163120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:56.178506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:56.180045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:56.201887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:56.203801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:56.205573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:56.205573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:56.205573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:12:56.205573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 371/751] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:56.205975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:56.219449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:56.220793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:56.221461 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:56.222087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:56.237413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:56.240198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:56.263829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:56.264824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:56.265741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:56.266369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 381/752] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:56.271688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:56.316406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:56.349485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:56.394722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ca04c-c408-495d-adc1-01a8967b7ad6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.473955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:56.475482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:56.476272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:56.477179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:56.479906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:56.480740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:56.483704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:56.491439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fd7c0-97f9-4034-afe3-b374ffe34dcb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.514188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 372/753] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:56.516372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:56.568198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:56.569624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:56.570356 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:56.571057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:56.573125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:56.573923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:56.576940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:56.602482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 382/754] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:56.604954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:56.716917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:56.730192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b4e4-9f21-4d2f-8e68-41030144c3bf None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.747816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780eb0-bb0d-4476-9541-879b5fcb0fa7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.790443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:56.791385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:56.791835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:56.792310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:56.797499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:56.798630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:56.799127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:56.799614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:56.801506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:56.802092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:56.802542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:56.804300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:56.804869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:12:56.819861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:56.820937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:12:56.821979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:56.826885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 383/755] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:12:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:56.828127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:56.841525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 373/756] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:56.850723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:56.856244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:56.951391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de30086-73c8-4d1b-980d-96300a7c2f49 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.975814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c5372-5847-44e4-a231-3c212c2aee2f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:56.991005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:56.991829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:56.992239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:56.992747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:56.996653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:57.001893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:57.004112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:57.016885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:57.017855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:57.018792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 384/757] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:57.020105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:57.034022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:57.034022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.034022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.034022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.034456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:57.034456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:57.034456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:57.060293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 374/758] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:12:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:57.061470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:57.158573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:57.175752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688376-a937-4ce2-a9b1-d7653ac9868c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:57.202612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1913-e9e4-4605-bb86-8f41b73d1d12 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:57.243413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:57.245539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.246453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.247192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.248763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=36c4eeb768c04233bcbfc786b3ba20b3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:57.249609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:57.252670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:57.264883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 385/759] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:57 2022] GET /identity/v3/role_assignments?user.id=36c4eeb768c04233bcbfc786b3ba20b3&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:57.266925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:57.285107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70af332-a28c-4fa5-ad02-3e2b62c8219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:57.286576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70af332-a28c-4fa5-ad02-3e2b62c8219f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.287005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70af332-a28c-4fa5-ad02-3e2b62c8219f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.287425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70af332-a28c-4fa5-ad02-3e2b62c8219f None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.288627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70af332-a28c-4fa5-ad02-3e2b62c8219f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:57.289292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70af332-a28c-4fa5-ad02-3e2b62c8219f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:57.292574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70af332-a28c-4fa5-ad02-3e2b62c8219f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:57.321479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 375/760] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:57.321479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba799df6-70bc-43ab-99bd-0d9b69b62427 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.321479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba799df6-70bc-43ab-99bd-0d9b69b62427 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.321479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba799df6-70bc-43ab-99bd-0d9b69b62427 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.323290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 376/761] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:57.327557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba799df6-70bc-43ab-99bd-0d9b69b62427 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:57.392109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:57.449440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229e2ff-c350-45d2-8047-750fe20f4889 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:57.499243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd30eb-cf1f-4a7c-bb33-462aec36673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:57.500171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd30eb-cf1f-4a7c-bb33-462aec36673a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.500608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd30eb-cf1f-4a7c-bb33-462aec36673a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.501227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd30eb-cf1f-4a7c-bb33-462aec36673a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.518015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd30eb-cf1f-4a7c-bb33-462aec36673a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:57.520600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd30eb-cf1f-4a7c-bb33-462aec36673a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:57.541122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba799df6-70bc-43ab-99bd-0d9b69b62427 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-43cd30eb-cf1f-4a7c-bb33-462aec36673a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:12:57.557486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:57.559521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:12:57.562394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:12:57.562394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:12:57.562394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:57.562394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 386/762] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:57.562859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a3ef2-8aa8-45dd-9978-05e29149d36b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.562992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a3ef2-8aa8-45dd-9978-05e29149d36b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.563387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a3ef2-8aa8-45dd-9978-05e29149d36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.595070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-633a3ef2-8aa8-45dd-9978-05e29149d36b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:57.597018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-633a3ef2-8aa8-45dd-9978-05e29149d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:57.600628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:57.618084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:57.618714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.618991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.619801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.620733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=a9c450c65b6849feb8c4250a1ca09fda) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:57.621098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:57.623212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:57.630962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 377/763] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:12:57 2022] GET /identity/v3/role_assignments?user.id=a9c450c65b6849feb8c4250a1ca09fda&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:57.631842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:57.792044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 387/764] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:12:57.839003 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95c8c5-7264-4fa6-8752-3c1c9de0af71 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:57.888666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:57.889304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:57.889641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:57.889940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:57.890655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:57.890958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:57.892386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:57.906628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 378/765] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:57.912686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a3ef2-8aa8-45dd-9978-05e29149d36b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:57.965889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:58.033590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a3ef2-8aa8-45dd-9978-05e29149d36b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:58.074307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:58.074883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:58.075160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:58.075428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:58.083602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:58.085126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:12:58.098164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:58.100328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:58.102524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 388/766] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:12:58.106795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:58.173115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:58.258251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104fcd4-b0ff-4e16-8e40-02fe1bad7a42 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:58.284900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:58.347728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:58.348622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:58.348911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:58.350239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:58.351007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:58.351318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:58.352829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:58.382179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 379/767] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:58.390744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:58.489187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b11ad5-919c-4117-b05e-303117a91eb4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:58.491095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:58.578804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:58.579475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:58.579756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:58.580028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:58.580705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:58.582540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:12:58.594423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:12:58.595504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:12:58.596360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 389/768] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 24 14:12:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:12:58.614155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86df92-4c4c-47e7-80d5-49c1670da5fe None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:58.615840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:58.624386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:58.660676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c433583-5130-4d48-bd6a-b95334630e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:58.662892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433583-5130-4d48-bd6a-b95334630e73 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:58.663509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433583-5130-4d48-bd6a-b95334630e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:58.664041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c433583-5130-4d48-bd6a-b95334630e73 None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/613e985e94c54913a5e6b93367157fc0/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:58.667076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c433583-5130-4d48-bd6a-b95334630e73 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=613e985e94c54913a5e6b93367157fc0, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:58.668937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c433583-5130-4d48-bd6a-b95334630e73 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:58.753271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 380/769] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:12:58 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/613e985e94c54913a5e6b93367157fc0/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:12:58.756879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f17e9-8fe6-4d60-a16e-4c21cc56151f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:58.757743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f17e9-8fe6-4d60-a16e-4c21cc56151f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:58.758310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f17e9-8fe6-4d60-a16e-4c21cc56151f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:58.766971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 381/770] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:12:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:12:58.771088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802e71c-a0dd-4be8-9d2a-d32eaa9e7e58 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:58.772115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802e71c-a0dd-4be8-9d2a-d32eaa9e7e58 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:58.772991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802e71c-a0dd-4be8-9d2a-d32eaa9e7e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:58.814082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a802e71c-a0dd-4be8-9d2a-d32eaa9e7e58 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:12:58.816845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a802e71c-a0dd-4be8-9d2a-d32eaa9e7e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:12:58.914182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fab86a-7f95-4200-88ed-dc5d8d7d3ec7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:58.984085 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 382/771] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:12:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:12:59.041479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:59.042720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:59.043039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:59.043761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:59.044965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:59.045506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:59.048289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:59.066224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 390/772] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 24 14:12:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:12:59.072342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802e71c-a0dd-4be8-9d2a-d32eaa9e7e58 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:59.225041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:59.235212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802e71c-a0dd-4be8-9d2a-d32eaa9e7e58 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:59.293553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:59.294218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:59.294483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:59.294757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:59.295476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:59.295779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:59.297337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:59.307490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 383/773] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 24 14:12:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:59.314025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:59.401085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:59.562676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1f3f9-3362-4961-8001-ec9bf9025612 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:59.616413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:59.617462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:59.617981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:59.618508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:59.619538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] RBAC: Adding query filter params (interface=public, service_id=e149303c18314a95a5db54086b0cb49b, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:12:59.620092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:59.621747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:59.632644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 391/774] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:12:59 2022] GET /identity/v3/endpoints?service_id=e149303c18314a95a5db54086b0cb49b&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:59.645537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:59.707873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:12:59.832343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31eda34-e9a0-4d2d-aa1f-c69e94309b16 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:59.899471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:59.900109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:59.901104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:59.901479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:59.902355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:59.903786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:12:59.912896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 384/775] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:12:59 2022] GET /identity/v3/services => generated 772 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:12:59.922410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:12:59.925302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead7836-7dbe-49b6-9cce-3c8d98c6e0cd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:12:59.981144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:12:59.981819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:12:59.982126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:12:59.982402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] PATH_INFO: `/v3/services/network` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:12:59.983010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:12:59.984440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:00.002579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:00.003702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:13:00.004596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:00.005514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 392/776] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:12:59 2022] GET /identity/v3/services/network => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:00.014386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:00.063671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:00.186034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eb3f7-b583-48d9-bd44-3d8fe4c467f8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:00.241115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df4da3-77c7-41a1-bcdf-c1445fa64a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:00.242485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df4da3-77c7-41a1-bcdf-c1445fa64a31 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:00.243034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df4da3-77c7-41a1-bcdf-c1445fa64a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:00.243561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df4da3-77c7-41a1-bcdf-c1445fa64a31 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:00.244948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df4da3-77c7-41a1-bcdf-c1445fa64a31 None admin] RBAC: Adding query filter params (name=network) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:00.245499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df4da3-77c7-41a1-bcdf-c1445fa64a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:00.248112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df4da3-77c7-41a1-bcdf-c1445fa64a31 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:00.257536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 385/777] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:00.266499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:00.345875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1d5ad-aad1-4548-b7f9-2cb525fb70b9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:00.346679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1d5ad-aad1-4548-b7f9-2cb525fb70b9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:00.347178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1d5ad-aad1-4548-b7f9-2cb525fb70b9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:00.358264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 386/778] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:00.360686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c090f-b858-4232-8b0e-f4efa5099f8b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:00.361185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c090f-b858-4232-8b0e-f4efa5099f8b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:00.361665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c090f-b858-4232-8b0e-f4efa5099f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:00.421212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-759c090f-b858-4232-8b0e-f4efa5099f8b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:00.423442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-759c090f-b858-4232-8b0e-f4efa5099f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:00.531585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2d7bb-6937-4b74-a408-ad04495be14e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:00.568554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 387/779] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:00.570951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a965-7e7d-42b6-aec6-6e91afd0b151 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:00.571413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a965-7e7d-42b6-aec6-6e91afd0b151 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:00.571854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a965-7e7d-42b6-aec6-6e91afd0b151 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:00.575478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 388/780] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:00.581178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b36b0-dc42-4d5e-84ab-319ed291ed22 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:00.581505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b36b0-dc42-4d5e-84ab-319ed291ed22 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:00.581719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b36b0-dc42-4d5e-84ab-319ed291ed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:00.599874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:00.600505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:00.600780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:00.601049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:00.601693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] RBAC: Adding query filter params (type=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:00.602016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:00.603550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:00.611020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 393/781] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:00.611461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6a4b36b0-dc42-4d5e-84ab-319ed291ed22 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:00.613581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6a4b36b0-dc42-4d5e-84ab-319ed291ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:00.623348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:00.662540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:00.688277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:00.758052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 389/782] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:00.872201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c53d9f-a88a-45ef-80e6-bb0a02a3d106 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:00.942143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:00.943205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:00.943696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:00.944182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:00.945498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] RBAC: Adding query filter params (interface=public, service_id=ff4f85300d0c4b019c01633deaf255cc, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:00.946109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:00.949439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:00.960214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 394/783] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:13:00 2022] GET /identity/v3/endpoints?service_id=ff4f85300d0c4b019c01633deaf255cc&interface=public®ion_id=RegionOne => generated 194 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:00.970124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b36b0-dc42-4d5e-84ab-319ed291ed22 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:01.045027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:01.220213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b36b0-dc42-4d5e-84ab-319ed291ed22 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:01.250452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:01.267472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:01.268726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:01.269210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:01.269634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:01.270494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:01.272061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:01.279054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 390/784] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:13:00 2022] GET /identity/v3/services => generated 772 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:01.286118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7a961-ac02-469a-b8e4-dcf5aeccd740 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:01.291937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:01.314061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:01.439294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:01.440403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:01.440878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:01.441368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:01.445512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2aefb-6797-4d88-b1cc-02e57aafc1a2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:01.483928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:01.493439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:01.495134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:01.495639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:01.496114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:01.502200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:01.503247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:01.504101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:01.505558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 395/785] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:01.511042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:01.515090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:01.520520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:01.521521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:13:01.522776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:01.531007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 391/786] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:13:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:01.534482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:01.632145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:01.693377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45e879-cc3d-44c4-ade3-eb610b43cca7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:01.752813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0d38e-e4cb-4596-a5a8-41a01cfb1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:01.754264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0d38e-e4cb-4596-a5a8-41a01cfb1c88 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:01.754844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0d38e-e4cb-4596-a5a8-41a01cfb1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:01.755874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0d38e-e4cb-4596-a5a8-41a01cfb1c88 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:01.757561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0d38e-e4cb-4596-a5a8-41a01cfb1c88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:01.758601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0d38e-e4cb-4596-a5a8-41a01cfb1c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:01.762376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0d38e-e4cb-4596-a5a8-41a01cfb1c88 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:01.769947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0c613-87e8-4d84-ad6b-3c1f59c99415 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:01.780875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 392/787] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:13:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:01.784138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1677bd5-0ece-480e-9d60-b9ecf6cb4c91 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:01.784948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1677bd5-0ece-480e-9d60-b9ecf6cb4c91 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:01.785746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1677bd5-0ece-480e-9d60-b9ecf6cb4c91 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:01.788962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 393/788] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:01.791535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1677bd5-0ece-480e-9d60-b9ecf6cb4c91 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:01.819660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59779c92-dbbe-4198-b4f7-e3a145e7df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:01.820368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59779c92-dbbe-4198-b4f7-e3a145e7df8d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:01.820676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59779c92-dbbe-4198-b4f7-e3a145e7df8d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:01.820963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59779c92-dbbe-4198-b4f7-e3a145e7df8d None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:01.821865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59779c92-dbbe-4198-b4f7-e3a145e7df8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:01.822348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59779c92-dbbe-4198-b4f7-e3a145e7df8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:01.823889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59779c92-dbbe-4198-b4f7-e3a145e7df8d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:01.836042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 396/789] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:13:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:01.837452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6b084-4d54-4e9a-bbbf-9fd6a288c4f8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:01.837758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6b084-4d54-4e9a-bbbf-9fd6a288c4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:01.838089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6b084-4d54-4e9a-bbbf-9fd6a288c4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:01.862622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-07b6b084-4d54-4e9a-bbbf-9fd6a288c4f8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:01.863866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-07b6b084-4d54-4e9a-bbbf-9fd6a288c4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:01.866358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:01.962127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 397/790] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:01.965119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6b084-4d54-4e9a-bbbf-9fd6a288c4f8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:02.048958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1677bd5-0ece-480e-9d60-b9ecf6cb4c91 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:02.173306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:02.174403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:02.174837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:02.175246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:02.190392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:02.215789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:02.218645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:02.234106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6b084-4d54-4e9a-bbbf-9fd6a288c4f8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:02.249520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:02.251670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:02.253440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:02.253440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:02.253440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:02.259427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 394/791] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:02.268744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:02.325819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:02.326634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:02.326769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:02.327044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:02.336161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:02.341940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:02.343677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:02.361436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:02.363728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:02.365650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:02.365650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:02.365650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:02.365650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 398/792] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:02.366934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:02.446533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4098df-df7f-488d-a361-15460e341367 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:02.493822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:02.494861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:02.495339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:02.495717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:02.509459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:02.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:02.518931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:02.520717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:02.525469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 395/793] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:02.526920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:02.592096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abc975-5d82-4ba4-a3fb-ea38e4e39a77 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:02.624988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:02.643288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:02.644107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:02.644484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:02.644892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:02.645926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:02.646138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:02.647637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:02.672282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 399/794] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:02.674711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:02.747589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe875-00db-4107-a6e6-8a5acddba22f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:02.748453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:02.794690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:02.795324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:02.795645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:02.795926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:02.796536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:02.796842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:02.798320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:02.814425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 396/795] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:02.815312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:02.858415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf776-6c8a-4df2-8697-6132c2365dba None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:02.909976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3551f6-5898-4991-b745-956583a11242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:02.910646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3551f6-5898-4991-b745-956583a11242 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:02.910994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3551f6-5898-4991-b745-956583a11242 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:02.911266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3551f6-5898-4991-b745-956583a11242 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:02.912337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3551f6-5898-4991-b745-956583a11242 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:02.912646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3551f6-5898-4991-b745-956583a11242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:02.914385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3551f6-5898-4991-b745-956583a11242 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:02.929283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 400/796] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:13:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:02.932529 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befd965-1d6c-4028-a5c3-2ca87d6e5918 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:02.933037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befd965-1d6c-4028-a5c3-2ca87d6e5918 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:02.933532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befd965-1d6c-4028-a5c3-2ca87d6e5918 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:02.936277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 401/797] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:02.937604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befd965-1d6c-4028-a5c3-2ca87d6e5918 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:03.023781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36940da2-22ea-4247-bff4-7d706ff86889 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:03.036812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:03.074328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:03.075303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.075743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.076151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.086259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:03.089070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:03.105824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:03.107753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 397/798] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:13:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:03.110532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:03.138287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befd965-1d6c-4028-a5c3-2ca87d6e5918 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:03.168042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:03.212428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849922a3-501c-4e25-946b-5d5451ecdbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:03.214337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849922a3-501c-4e25-946b-5d5451ecdbea None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.214828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849922a3-501c-4e25-946b-5d5451ecdbea None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.215293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849922a3-501c-4e25-946b-5d5451ecdbea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.231559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849922a3-501c-4e25-946b-5d5451ecdbea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:03.234481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849922a3-501c-4e25-946b-5d5451ecdbea None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-849922a3-501c-4e25-946b-5d5451ecdbea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:03.257719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:03.259733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:13:03.262023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:03.267760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556659a2-3043-4faf-841c-fedeab78f864 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:03.268695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 402/799] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:03.271126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938ead3-d1d2-403b-88bf-7962580f8aa6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.271907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938ead3-d1d2-403b-88bf-7962580f8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.272399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938ead3-d1d2-403b-88bf-7962580f8aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.305588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7938ead3-d1d2-403b-88bf-7962580f8aa6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:03.308646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7938ead3-d1d2-403b-88bf-7962580f8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:03.318340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:03.319681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.320185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.320656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.336303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:03.338893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:03.359322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:03.360364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:03.361361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:03.361361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:03.361361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:03.368875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 398/800] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:03.370252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:03.409822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:03.434027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 403/801] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:03.434027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938ead3-d1d2-403b-88bf-7962580f8aa6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:03.593849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc66b4f-666e-46ab-9f1e-8881aed695ed None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:03.641599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938ead3-d1d2-403b-88bf-7962580f8aa6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:03.644153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:03.669392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:03.670643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.671219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.671777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.673366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:03.674029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:03.676951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:03.687799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:03.688758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.689200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.689894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.691066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:03.692020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:03.693643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:03.715655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 404/802] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 24 14:13:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:03.716988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:03.718381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 399/803] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 24 14:13:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:03.724903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:03.854506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:03.893032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7675df-bee7-4791-a765-071a987a17d3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:03.893941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f7d5b-a82d-4d45-b131-0aa2d0e91b89 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:03.940471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de023efd-694c-42fb-87b1-a52228da684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:03.941225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023efd-694c-42fb-87b1-a52228da684b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.941603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023efd-694c-42fb-87b1-a52228da684b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.941770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:03.942023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023efd-694c-42fb-87b1-a52228da684b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.942586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.943008 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.943343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.944080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:03.944486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:03.946343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:03.956293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de023efd-694c-42fb-87b1-a52228da684b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:03.958118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de023efd-694c-42fb-87b1-a52228da684b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:03.962050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 405/804] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:03.963933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-de023efd-694c-42fb-87b1-a52228da684b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:03.979487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:03.981651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:03.983497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:03.983497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:03.983497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:03.983497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 400/805] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:03.984800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10221a49-6b6e-4e74-bd4e-feddd3a345ba None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:03.984800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10221a49-6b6e-4e74-bd4e-feddd3a345ba None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:03.985100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10221a49-6b6e-4e74-bd4e-feddd3a345ba None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:03.987562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 401/806] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:03.989636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10221a49-6b6e-4e74-bd4e-feddd3a345ba None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:04.001224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:04.121553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10221a49-6b6e-4e74-bd4e-feddd3a345ba None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:04.128185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dfae6-ccd8-4edf-81e5-e592010203ad None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:04.158469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:04.173114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:04.174262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.178242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.178242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.178242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:04.178935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:04.179523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:04.180285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.180713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.181084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.194530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:04.196030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:04.196203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:04.198895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:04.200834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 402/807] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 24 14:13:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:04.202815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:04.222224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:04.224120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:04.225883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:04.225883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:04.225883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:04.226304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 406/808] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:04.228731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:04.324940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272333d-d816-4d74-8748-04c097a96e06 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:04.366612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f389b3-1fd2-41e8-a025-2da7fa7af629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:04.366970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f389b3-1fd2-41e8-a025-2da7fa7af629 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.367245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f389b3-1fd2-41e8-a025-2da7fa7af629 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.367530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f389b3-1fd2-41e8-a025-2da7fa7af629 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.377897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f389b3-1fd2-41e8-a025-2da7fa7af629 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:04.379461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f389b3-1fd2-41e8-a025-2da7fa7af629 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-81f389b3-1fd2-41e8-a025-2da7fa7af629 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:04.392660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:04.400795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 403/809] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:13:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35415257-66f4-4717-8b47-86fca873f147 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.402713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35415257-66f4-4717-8b47-86fca873f147 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.404050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35415257-66f4-4717-8b47-86fca873f147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.416115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-35415257-66f4-4717-8b47-86fca873f147 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:04.417301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-35415257-66f4-4717-8b47-86fca873f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:04.446798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:04.474556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:04.496306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955626ac-5a8f-4ae2-87e7-50850eb02ef3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:04.502409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 404/810] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:04.503568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35415257-66f4-4717-8b47-86fca873f147 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:04.573267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:04.575040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.575827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.576571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.578485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:04.579515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:04.582804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:04.615735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 407/811] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:04.615735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:04.668935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35415257-66f4-4717-8b47-86fca873f147 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:04.735761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:04.736505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.736784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.737064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.737702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:04.738070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:04.739526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:04.745628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 405/812] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 24 14:13:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:04.746580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:04.793581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c664-1f2e-41fb-9c6a-3651267cd0dd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:04.821373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:04.851367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:04.860276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:04.861427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.861827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.862350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.863553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:04.864078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:04.866798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:04.889769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 408/813] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:04.891710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:04.901584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0e9fa-bde7-41fa-933d-5a9e1281163e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:04.945937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:04.946937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:04.947983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:04.948483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:04.949858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:04.950420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:04.953189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:04.987275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 406/814] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:13:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:04.988488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.023725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b24c7f-fb6b-4b63-a1d4-5099e6e9c23c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.065566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.066258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.066533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.066873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.076173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.077822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:05.090707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:05.091769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:05.092653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 409/815] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:05.093144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.113532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b7128-6bdb-4d3f-ab17-30f2501f0d94 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.116574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:05.150863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.151475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.151957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.152229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.152859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:05.153157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.154463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:05.154630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.160551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 407/816] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 24 14:13:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:05.161690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.206936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b093-cdcd-49c6-88b3-910454adbe26 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.246315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.246963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.247244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.247516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] PATH_INFO: `/v3/services/network` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.248112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.249538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:05.258848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:05.261029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:05.262776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 410/817] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:05 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:05.264205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.286668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766d3b0-f6cb-4280-8c22-65c4a493cb8c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.325088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.325801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.326039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.326308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.334654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.336157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:05.349004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:05.350898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:05.352750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 408/818] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:05.353782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.383806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de3ec8-24b0-4e25-8c3c-7fd8d1e4b030 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.395891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:05.404084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:05.425579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.426233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.426500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.426768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.491510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e45119-d3f9-4fcf-8e34-f9018dfeb7d4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.514749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.516658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.543124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.543740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.544012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.544277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.544976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:05.545275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.546791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.559423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 409/819] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:05.560280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.675227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 411/820] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:13:05.676797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.694349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:05.702628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:05.713877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409b9f8-7e7e-4023-8205-40ba57cefe29 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.759296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.760110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.760595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.760955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.762065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.764460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.818115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72772814-d5e9-4ea5-bc98-606e06994db4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:05.859882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:05.860502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:05.860781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:05.861054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:05.861695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:05.862030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:05.863536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:05.869261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 412/821] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:05.871142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:05.885778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 410/822] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:05.885778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:06.015191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:06.022563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9f40b-7ec1-4a57-a9b5-9c5b2f85f65e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:06.048582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dccf7b-8ba8-47a4-b108-c45360debcd6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:06.079756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:06.080465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:06.080729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:06.081008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/36c4eeb768c04233bcbfc786b3ba20b3/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:06.083763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=36c4eeb768c04233bcbfc786b3ba20b3, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:06.085373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:06.101971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:06.102733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:06.103082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:06.104745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:06.105527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:06.105907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:06.107405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:06.121561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 413/823] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:06.122588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:06.151914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:06.164317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 411/824] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:13:05 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/36c4eeb768c04233bcbfc786b3ba20b3/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:13:06.165868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:06.336515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65695aaa-88cb-4371-a9ca-e7ff461a6f63 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:06.383796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a42cf-d678-481d-beac-66721747834b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:06.421782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:06.423213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:06.423908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:06.424556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:06.442384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:06.447686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:06.450899 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:06.455629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:06.456682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:06.457167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:06.457615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:06.458747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] RBAC: Adding query filter params (type=network) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:06.460475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:06.462295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:06.481045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:06.483061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:06.484874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:06.484874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:06.484874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:06.485225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 412/825] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:06.485363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 414/826] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:06.485487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:06.499585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:06.634893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:06.728495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7b44b-14dd-4648-9a58-3c25d0377774 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:06.760868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062d428-1031-48c5-b780-d93882808be0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:06.801705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:06.803402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:06.804193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:06.805012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/a9c450c65b6849feb8c4250a1ca09fda/roles/adf17858daf74c2ea9cc93775172b567` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:06.829632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:06.830861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:06.831344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:06.831827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:06.833197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:06.833573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:06.836287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:06.839124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=a9c450c65b6849feb8c4250a1ca09fda, role_id=adf17858daf74c2ea9cc93775172b567)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:06.842550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:06.860071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 415/827] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:06.862362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:06.940927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 413/828] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:13:06 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/a9c450c65b6849feb8c4250a1ca09fda/roles/adf17858daf74c2ea9cc93775172b567 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:13:06.943171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:07.026638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:07.067419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dc74e-311a-4fa5-8e18-cdc3a47e9f71 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:07.132405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:07.133475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:07.133958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:07.134547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:07.136017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:07.138777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:07.199897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dfa7d-0a90-46e4-ae7c-b686d6f86e9d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:07.234860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:07.274680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 416/829] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:06 2022] POST /identity/v3/endpoints => generated 337 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:07.279881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:07.281083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:07.281636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:07.282428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:07.300742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:07.303700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:07.328490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:07.329647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:07.330581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:07.331478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 414/830] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:07.341894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:07.593913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e810cd3-54da-4d23-b3b7-1855b5ee311e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:07.608381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:07.657830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b124bb0-cdb8-434d-a903-11541bc790d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:07.658947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124bb0-cdb8-434d-a903-11541bc790d1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:07.659429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124bb0-cdb8-434d-a903-11541bc790d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:07.659887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124bb0-cdb8-434d-a903-11541bc790d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:07.661178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124bb0-cdb8-434d-a903-11541bc790d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:07.661708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124bb0-cdb8-434d-a903-11541bc790d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:07.664593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124bb0-cdb8-434d-a903-11541bc790d1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:07.697426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 417/831] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:07.704805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:07.910717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd55c01-8195-4dd9-b2c2-884059614863 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:07.952359 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b0c44-542d-42ba-81f6-e5c7200309a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:07.953392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0c44-542d-42ba-81f6-e5c7200309a3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:07.953882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0c44-542d-42ba-81f6-e5c7200309a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:07.954391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0c44-542d-42ba-81f6-e5c7200309a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:07.955363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b0c44-542d-42ba-81f6-e5c7200309a3 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=613e985e94c54913a5e6b93367157fc0) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:07.955911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b0c44-542d-42ba-81f6-e5c7200309a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:07.957597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b0c44-542d-42ba-81f6-e5c7200309a3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:07.965500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 415/832] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:07 2022] GET /identity/v3/role_assignments?user.id=613e985e94c54913a5e6b93367157fc0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:08.894863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96685fa4-d374-4dd8-8d2e-b9ebc69df8e0 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:08.895409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96685fa4-d374-4dd8-8d2e-b9ebc69df8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:08.895409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96685fa4-d374-4dd8-8d2e-b9ebc69df8e0 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:08.897088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 418/833] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:08.903156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16637-0ba0-4906-be61-ad6346546129 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:08.903679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16637-0ba0-4906-be61-ad6346546129 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:08.903679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16637-0ba0-4906-be61-ad6346546129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:08.923652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a5e16637-0ba0-4906-be61-ad6346546129 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:08.924906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a5e16637-0ba0-4906-be61-ad6346546129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:08.958581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:09.010772 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 416/834] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:09.070355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5db830-2b09-4e15-8582-966a2d084f0f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:09.070852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5db830-2b09-4e15-8582-966a2d084f0f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:09.071495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5db830-2b09-4e15-8582-966a2d084f0f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:09.075132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 419/835] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:09.083498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0be77-8489-4271-9fe1-60049b7fa3b8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:09.083761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0be77-8489-4271-9fe1-60049b7fa3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:09.084081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0be77-8489-4271-9fe1-60049b7fa3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:09.104619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-57a0be77-8489-4271-9fe1-60049b7fa3b8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:09.105891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-57a0be77-8489-4271-9fe1-60049b7fa3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:09.185663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 417/836] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:09.295690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5db830-2b09-4e15-8582-966a2d084f0f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:09.429548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5db830-2b09-4e15-8582-966a2d084f0f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:09.467091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0be77-8489-4271-9fe1-60049b7fa3b8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:09.473301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:09.473963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:09.474271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:09.474551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:09.491095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:09.499034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:09.500067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:09.500891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 420/837] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:09.510146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:09.549903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:09.596789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:09.606565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0be77-8489-4271-9fe1-60049b7fa3b8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:09.653036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8569d1-302d-4a58-a640-baab72257971 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:09.658288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:09.659369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:09.659851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:09.660354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:09.661169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:09.662857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:09.676056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:09.677100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:09.677959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:09.679423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 418/838] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:09.690294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:09.710380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:09.711490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:09.711773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:09.712045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:09.713099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:09.713419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:09.715696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:09.730185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 421/839] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:13:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:09.735023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:09.833172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221265fa-9561-4d5b-8af5-6cdeb699b173 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:09.867780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:09.880459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:09.881722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:09.881837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cffd4-3e23-4d10-96dc-529a1d22afb9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:09.881951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:09.881951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:09.882297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:09.882606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:09.886097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:09.894256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 419/840] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:09.899370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:09.936795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:09.938306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:09.938687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:09.939059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:09.953926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:09.956118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:09.966473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:09.977132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:09.979474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:09.980693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:09.980693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:09.980693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:09.982114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 422/841] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:09.987057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:10.045527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc66fc2-0364-4a35-997b-2874d42ffaec None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:10.085213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9265a-00ce-43c8-883c-ebfa89cf52f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:10.085846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9265a-00ce-43c8-883c-ebfa89cf52f0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.086166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9265a-00ce-43c8-883c-ebfa89cf52f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.086459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9265a-00ce-43c8-883c-ebfa89cf52f0 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.087104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9265a-00ce-43c8-883c-ebfa89cf52f0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:10.087477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9265a-00ce-43c8-883c-ebfa89cf52f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:10.089374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9265a-00ce-43c8-883c-ebfa89cf52f0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:10.098398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 420/842] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:10.135505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aeed37-ddc2-41c4-8659-bba3eba0b535 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:10.155674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2300a-a51e-41fd-aeb2-cb14376d390a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.156051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2300a-a51e-41fd-aeb2-cb14376d390a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.156244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2300a-a51e-41fd-aeb2-cb14376d390a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.158293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 421/843] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:10.164227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722440-daec-4427-bf7c-b3ba69d57ec2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.164605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722440-daec-4427-bf7c-b3ba69d57ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.164796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722440-daec-4427-bf7c-b3ba69d57ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.185591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7a722440-daec-4427-bf7c-b3ba69d57ec2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:10.186913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7a722440-daec-4427-bf7c-b3ba69d57ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:10.188032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:10.189006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.189792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.190322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.192437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:10.193049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:10.194822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:10.215195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 423/844] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:10.221632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:10.226268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:10.322626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 422/845] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:10.344400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fe0e3-9085-4d90-8f06-f376df86bbf1 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.344747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fe0e3-9085-4d90-8f06-f376df86bbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.344945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fe0e3-9085-4d90-8f06-f376df86bbf1 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.346744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 423/846] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:10.356460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56d121-b24a-42c3-8c5c-2365e5fb8df4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.356885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56d121-b24a-42c3-8c5c-2365e5fb8df4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.357262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56d121-b24a-42c3-8c5c-2365e5fb8df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.379715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec03b2-572f-407c-aa49-4b9268cc0b74 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:10.386214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ac56d121-b24a-42c3-8c5c-2365e5fb8df4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:10.387696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ac56d121-b24a-42c3-8c5c-2365e5fb8df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:10.401791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:10.425805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:10.426764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.427143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.427514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.437447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:10.439868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:10.453318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:10.454625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 14:13:10.455804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:10.457588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 424/847] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:10.464012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:10.484764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 424/848] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:10.544861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:10.593127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee30d75-1281-44e7-a9ea-6a631ad39bdc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:10.635261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:10.636029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.636328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.636617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.637522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:10.637734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:10.639410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:10.661150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 425/849] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 24 14:13:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:10.661474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56d121-b24a-42c3-8c5c-2365e5fb8df4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:10.669696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:10.811872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56d121-b24a-42c3-8c5c-2365e5fb8df4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:10.830962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc7ca0-4609-41ec-9886-1f56531ff929 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:10.851293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:10.851881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.852161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.852436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:10.853851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:10.855065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:10.863478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:10.865555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:10.880567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:10.882532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 425/850] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 24 14:13:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:10.884115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:10.885878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:10.902747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:10.903773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:10.904449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:10.904897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.024868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a6b4-181d-4a7d-b63f-0aceb54840f0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:11.026599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:11.028091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:11.083656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:11.084677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:11.085111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:11.085527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.100517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:11.110737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:11.112457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:11.113409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 426/851] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:11.116182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.191199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 426/852] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 14:13:11.191199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.209249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:11.220990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:11.270748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc17056-1600-461f-b608-9626fa5f097b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:11.327866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477064b-377a-4c84-9c46-015ed6028ee5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:11.347238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:11.347858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:11.348159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:11.348430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.349069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:11.349566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:11.350855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:11.357796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 427/853] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:11.358917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.395077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:11.395966 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:11.396300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:11.396634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.397632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:11.398318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:11.401363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:11.418663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 427/854] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:11.419983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.526719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9b664-a465-4105-9a6c-f5cf7c08bcb3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:11.573438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:11.583282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:11.584241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:11.592843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:11.592843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.592843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:11.604039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbed70-28c0-4bf3-be5d-a9197ae9adf8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:11.604658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:11.607818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:11.625850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:11.627805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:11.629558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:11.629558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:11.629558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:11.629558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 428/855] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:11.629558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.669857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:11.671052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:11.671599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:11.672079 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.673350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:11.674139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:11.676197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:11.694034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 428/856] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:11.694034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.864995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c306-6dcf-46dc-b6a1-1bce46dfa95f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:11.882019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395efc35-2da2-4421-ae06-d32f1a25b2f1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:11.923612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:11.924297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:11.924572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:11.924800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:11.924936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.925760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:11.926279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:11.926749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:11.927671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:11.928218 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:11.929802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:11.938821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:11.941111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:11.944200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 429/857] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:11.954032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:11.959053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:11.960072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:11.960946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:11.960946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:11.960946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:11.960946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 429/858] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:11.968011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:11.970505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:12.019695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:12.095930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1aab6-62f5-4725-9a85-b2e7ca9f903c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:12.103688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82add2f-c163-4c33-8780-cd4ce6e11fb2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:12.149018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:12.151616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.152176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.152932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.153964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:12.154517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:12.156500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:12.173002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:12.178061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.178061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.178061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.178640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 430/859] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:12.184819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:12.189845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:12.191778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:12.207495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:12.212954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:12.215089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 430/860] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:12.218694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:12.307253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:12.347095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:12.353786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba470e-f925-4fce-8ef7-b99eb9f26d5b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:12.356215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851537e-39e5-4297-91c5-a063f3911649 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:12.405543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712cb25-5cae-4d77-8b25-b7074208dddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:12.406141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712cb25-5cae-4d77-8b25-b7074208dddd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.406420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712cb25-5cae-4d77-8b25-b7074208dddd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.406686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712cb25-5cae-4d77-8b25-b7074208dddd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.407770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935b7f6-2c46-42d4-84ec-8b3919aab79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:12.408662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935b7f6-2c46-42d4-84ec-8b3919aab79f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.409132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935b7f6-2c46-42d4-84ec-8b3919aab79f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.409242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935b7f6-2c46-42d4-84ec-8b3919aab79f None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.409985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935b7f6-2c46-42d4-84ec-8b3919aab79f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:12.410336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935b7f6-2c46-42d4-84ec-8b3919aab79f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:12.411811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935b7f6-2c46-42d4-84ec-8b3919aab79f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:12.416317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712cb25-5cae-4d77-8b25-b7074208dddd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:12.418298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712cb25-5cae-4d77-8b25-b7074208dddd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:12.429920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 431/861] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:12.431401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be0feb-3b27-4db4-8b21-03f0015a0cda None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.431688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be0feb-3b27-4db4-8b21-03f0015a0cda None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.431962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be0feb-3b27-4db4-8b21-03f0015a0cda None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6712cb25-5cae-4d77-8b25-b7074208dddd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:12.432979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:12.435060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:12.435996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:12.437042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 432/862] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:12.438263 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be0feb-3b27-4db4-8b21-03f0015a0cda None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:12.443466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 431/863] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:13:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:12.443466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fc22d-94f9-4882-aa88-40c5eb4518e9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.443466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fc22d-94f9-4882-aa88-40c5eb4518e9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.443466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fc22d-94f9-4882-aa88-40c5eb4518e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.462564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bd7fc22d-94f9-4882-aa88-40c5eb4518e9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:12.464027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bd7fc22d-94f9-4882-aa88-40c5eb4518e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:12.576391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 432/864] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:12.577407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fc22d-94f9-4882-aa88-40c5eb4518e9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:12.606289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be0feb-3b27-4db4-8b21-03f0015a0cda None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:12.649496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:12.655151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:12.656542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.657236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.657663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.658635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=36c4eeb768c04233bcbfc786b3ba20b3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:12.659502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:12.666373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:12.678301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 433/865] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:12 2022] GET /identity/v3/role_assignments?user.id=36c4eeb768c04233bcbfc786b3ba20b3&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:12.744795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:12.767270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fc22d-94f9-4882-aa88-40c5eb4518e9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:12.826233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:12.827022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:12.827483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:12.827818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:12.828872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:12.829488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:12.831746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:12.865086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 433/866] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 24 14:13:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:12.871331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:12.877594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:13.021763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319122ca-ee95-4687-bc74-e3928f1c5884 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:13.031328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe870c9-1351-4bab-9d8e-be04ed25dcc5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:13.062614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:13.063248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:13.063522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:13.063787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:13.070917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:13.072158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:13.072490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:13.072804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:13.073423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:13.074934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:13.136854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:13.151257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:13.152829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:13.194268 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 434/867] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:13:12 2022] POST /identity/v3/services => generated 251 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:13:13.378446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 434/868] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:13:13.383245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:13.410434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:13.596530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa100-67e1-4437-9c61-02c13599d4e4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:13.648538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8deb9-c580-4c08-8389-75842db798a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:13.649597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8deb9-c580-4c08-8389-75842db798a8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:13.650797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8deb9-c580-4c08-8389-75842db798a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:13.651292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8deb9-c580-4c08-8389-75842db798a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:13.671739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8deb9-c580-4c08-8389-75842db798a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:13.674670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8deb9-c580-4c08-8389-75842db798a8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-70e8deb9-c580-4c08-8389-75842db798a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:13.701248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:13.702307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:13.703234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:13.703234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:13.703234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:13.710164 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 435/869] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:13.713276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:13.859941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:13.881897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8a1d3-da2f-42ea-913d-fdf35f67b79e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:13.925039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c4ded-e402-4e68-8b0e-04377044ac17 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:13.925221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c4ded-e402-4e68-8b0e-04377044ac17 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:13.925486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c4ded-e402-4e68-8b0e-04377044ac17 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:13.926904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 436/870] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:13.932951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:13.933589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:13.933870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:13.934175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:13.934662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa30ab5-af7f-418f-b457-687f3c825717 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:13.934860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:13.935142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:13.935573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa30ab5-af7f-418f-b457-687f3c825717 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:13.936085 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa30ab5-af7f-418f-b457-687f3c825717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:13.936545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:13.952058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 435/871] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:13.959503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:13.965087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3fa30ab5-af7f-418f-b457-687f3c825717 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:13.965668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3fa30ab5-af7f-418f-b457-687f3c825717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:14.027764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:14.054027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 437/872] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:14.087714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45782-4e90-4499-99d6-a91374eff924 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:14.126664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83516470-0940-4d50-959b-7fb40312732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:14.127312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83516470-0940-4d50-959b-7fb40312732a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:14.127584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83516470-0940-4d50-959b-7fb40312732a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:14.127852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83516470-0940-4d50-959b-7fb40312732a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:14.136190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83516470-0940-4d50-959b-7fb40312732a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:14.137651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83516470-0940-4d50-959b-7fb40312732a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-83516470-0940-4d50-959b-7fb40312732a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:14.150226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:14.151397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:14.152317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 436/873] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:14.159502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa30ab5-af7f-418f-b457-687f3c825717 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:14.276050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa30ab5-af7f-418f-b457-687f3c825717 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:14.319499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:14.320559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:14.321140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:14.321696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:14.323106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:14.323797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:14.325529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:14.331339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83516470-0940-4d50-959b-7fb40312732a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:14.342781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 438/874] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:14.346871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:14.349284 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:14.408878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:14.455543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83516470-0940-4d50-959b-7fb40312732a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:14.486836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f4173-2d4d-444b-b55f-76837ba1a49d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:14.499613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a8f09-5648-46ca-9f76-4d7fed75b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:14.501193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8f09-5648-46ca-9f76-4d7fed75b4fb None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:14.501739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8f09-5648-46ca-9f76-4d7fed75b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:14.502208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8f09-5648-46ca-9f76-4d7fed75b4fb None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:14.503061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a8f09-5648-46ca-9f76-4d7fed75b4fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:14.505428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a8f09-5648-46ca-9f76-4d7fed75b4fb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:14.534795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57355a8-28f9-4e32-9cfc-dbf6b5293d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:14.534795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57355a8-28f9-4e32-9cfc-dbf6b5293d58 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:14.534795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57355a8-28f9-4e32-9cfc-dbf6b5293d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:14.534795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57355a8-28f9-4e32-9cfc-dbf6b5293d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:14.535461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57355a8-28f9-4e32-9cfc-dbf6b5293d58 None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=a9c450c65b6849feb8c4250a1ca09fda) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:14.535461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57355a8-28f9-4e32-9cfc-dbf6b5293d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:14.535461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57355a8-28f9-4e32-9cfc-dbf6b5293d58 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:14.543212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 439/875] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:14 2022] GET /identity/v3/role_assignments?user.id=a9c450c65b6849feb8c4250a1ca09fda&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:14.602447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 437/876] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:13:14 2022] POST /identity/v3/services => generated 269 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:15.018201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daa6d5-5434-4973-9447-66a929458ab1 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:15.018479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daa6d5-5434-4973-9447-66a929458ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:15.018833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daa6d5-5434-4973-9447-66a929458ab1 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:15.020522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 440/877] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:15.028165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147deee0-4a5c-45a0-b5af-bf57cfac00e4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:15.028677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147deee0-4a5c-45a0-b5af-bf57cfac00e4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:15.029179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147deee0-4a5c-45a0-b5af-bf57cfac00e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:15.062761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-147deee0-4a5c-45a0-b5af-bf57cfac00e4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:15.064754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-147deee0-4a5c-45a0-b5af-bf57cfac00e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:15.190751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:15.223304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 438/878] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:15.479225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123c0a-1b73-4726-bba6-09c40070bd26 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:15.479793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123c0a-1b73-4726-bba6-09c40070bd26 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:15.479793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123c0a-1b73-4726-bba6-09c40070bd26 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:15.481546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 441/879] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:15.487380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e861fc7-79fc-4799-8741-da108486d516 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:15.487699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e861fc7-79fc-4799-8741-da108486d516 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:15.488019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e861fc7-79fc-4799-8741-da108486d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:15.507796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94123c0a-1b73-4726-bba6-09c40070bd26 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:15.517807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5e861fc7-79fc-4799-8741-da108486d516 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:15.519951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5e861fc7-79fc-4799-8741-da108486d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:15.651607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 439/880] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:15.670276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94123c0a-1b73-4726-bba6-09c40070bd26 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:15.684396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:15.714487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:15.715533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:15.716086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:15.716557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:15.717483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:15.719248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:15.729587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:15.730743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:15.731577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:15.734712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 442/881] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:15.741007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e861fc7-79fc-4799-8741-da108486d516 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:15.863122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e861fc7-79fc-4799-8741-da108486d516 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:15.893239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:15.904632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:15.905282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:15.905611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:15.905881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:15.906893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:15.907191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:15.913118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:15.914771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 440/882] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:15.921951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:15.946393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:16.046351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748df608-c77b-444b-8aa8-16c10821739d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:16.091342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:16.091768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704f592-2b96-4c36-9849-e6abc624a161 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:16.092144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:16.092492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:16.092830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:16.093612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:16.094007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:16.095635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:16.106160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 443/883] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:16.136390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:16.137099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:16.137375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:16.137643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:16.138301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:16.139798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:16.150314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:16.151818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:16.152671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 441/884] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:16.158633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:16.209052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:16.283324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6eb3f-e102-4131-9016-e95d3192593b None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:16.326425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:16.327060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:16.327334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:16.327608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:16.328289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:16.328597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:16.330041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:16.336260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 444/885] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:16.344283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:16.475711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:16.533524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416502f3-e820-4598-a50f-383054d2f917 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:16.574483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:16.575376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:16.575828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:16.576257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:16.577069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:16.577521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:16.579139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:16.585533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 442/886] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:16.595742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:16.801733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:16.805502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c58be-d827-47f3-87ab-4eefa7cc0abd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:16.851121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe873f7-2c3b-48a9-9036-cf85ddc5c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:16.852308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe873f7-2c3b-48a9-9036-cf85ddc5c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:16.853442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe873f7-2c3b-48a9-9036-cf85ddc5c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:16.853930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe873f7-2c3b-48a9-9036-cf85ddc5c6c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:16.855017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe873f7-2c3b-48a9-9036-cf85ddc5c6c4 None admin] RBAC: Adding query filter params (interface=public, service_id=dabcb57c06b44533a584f1c0bbed3b4f, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:16.855555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe873f7-2c3b-48a9-9036-cf85ddc5c6c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:16.857308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe873f7-2c3b-48a9-9036-cf85ddc5c6c4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:16.870468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 445/887] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:13:16 2022] GET /identity/v3/endpoints?service_id=dabcb57c06b44533a584f1c0bbed3b4f&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:16.873509 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:16.876428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8b52a-ac12-4b00-a73f-7fa1bb830050 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:16.877372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8b52a-ac12-4b00-a73f-7fa1bb830050 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:16.878125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8b52a-ac12-4b00-a73f-7fa1bb830050 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:16.880079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 446/888] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:16.890095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ca962-bb1d-4b2e-974a-d8bab0ab68eb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:16.890982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ca962-bb1d-4b2e-974a-d8bab0ab68eb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:16.891706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ca962-bb1d-4b2e-974a-d8bab0ab68eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:16.932133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0b5ca962-bb1d-4b2e-974a-d8bab0ab68eb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:16.936918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0b5ca962-bb1d-4b2e-974a-d8bab0ab68eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:17.030899 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994fd5c-cd5f-4f67-a3b4-9383b8006b78 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:17.048738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 447/889] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:17.050402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c2344-2b43-4e74-8d8e-9febb0b83e3e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:17.050841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c2344-2b43-4e74-8d8e-9febb0b83e3e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:17.051285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c2344-2b43-4e74-8d8e-9febb0b83e3e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:17.053146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 448/890] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:17.062720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579234d4-9bf1-496e-b3d8-e47b277d2686 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:17.063405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579234d4-9bf1-496e-b3d8-e47b277d2686 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:17.063913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579234d4-9bf1-496e-b3d8-e47b277d2686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:17.072644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:17.089191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-579234d4-9bf1-496e-b3d8-e47b277d2686 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:17.091069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-579234d4-9bf1-496e-b3d8-e47b277d2686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:17.099606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:17.100638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:17.101133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:17.101620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:17.102957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:17.105931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:17.125409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 443/891] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:13:16 2022] GET /identity/v3/services => generated 1268 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:13:17.189701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 449/892] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:17.365674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:17.488948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c781af-9b29-4160-ac78-f23de2d23c1a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:17.529422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:17.530534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:17.531030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:17.531502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:17.532558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:17.534182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:17.544333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:17.545349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:17.546317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:17.549234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 444/893] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 24 14:13:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:17.555559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579234d4-9bf1-496e-b3d8-e47b277d2686 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:17.585970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:17.677464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579234d4-9bf1-496e-b3d8-e47b277d2686 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:17.687841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:17.733129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:17.734253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:17.734733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:17.735251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:17.736309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:17.736825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:17.739295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:17.749113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 450/894] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:17.755171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:17.800575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:17.859725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:17.874618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e5774-2a7e-4693-9eb7-1fec674da8a3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:17.913254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e7d5-792e-4018-8312-ac48236cf686 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:17.916200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:17.916893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:17.917175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:17.917450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:17.918315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:17.918510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:17.919888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:17.933454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 451/895] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:17.934523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:17.987950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:17.989654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:17.990529 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:17.991378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:18.013942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:18.025231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:18.027154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:18.028798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:18.041685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 445/896] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 24 14:13:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:18.044128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:18.056356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cc32f-57a4-4a8f-804e-e5c76f396081 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:18.098574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:18.099191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:18.099469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:18.099743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:18.100471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] RBAC: Adding query filter params (interface=public, service_id=a8280fb5055e42dd941bfb1bda75d68b, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:18.100846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:18.102337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:18.113004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 452/897] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:13:17 2022] GET /identity/v3/endpoints?service_id=a8280fb5055e42dd941bfb1bda75d68b&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:18.114304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:18.136618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:18.235886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f009-d6c8-4b34-90b9-46443fcd0025 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:18.262815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeef5df-b495-427a-8c87-8d08c5e20dcb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:18.277148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:18.277937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:18.278352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:18.278719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:18.279346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:18.279516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:18.281434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:18.294576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 453/898] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:13:18 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:13:18.338709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f700f9a-a773-4591-a4ee-32600b61278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:18.340137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f700f9a-a773-4591-a4ee-32600b61278a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:18.340883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f700f9a-a773-4591-a4ee-32600b61278a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:18.341570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f700f9a-a773-4591-a4ee-32600b61278a None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:18.343144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f700f9a-a773-4591-a4ee-32600b61278a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:18.343951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f700f9a-a773-4591-a4ee-32600b61278a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:18.347352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f700f9a-a773-4591-a4ee-32600b61278a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:18.373713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 446/899] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 24 14:13:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:18.377702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:18.393088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598e827-c251-406d-8618-6b204bf032ce None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:18.393679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598e827-c251-406d-8618-6b204bf032ce None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:18.394021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598e827-c251-406d-8618-6b204bf032ce None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:18.395732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 447/900] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:18.401852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285acd9-26d7-4902-a3d8-5948e5d13956 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:18.402201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285acd9-26d7-4902-a3d8-5948e5d13956 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:18.402522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285acd9-26d7-4902-a3d8-5948e5d13956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:18.424445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3285acd9-26d7-4902-a3d8-5948e5d13956 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:18.425739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3285acd9-26d7-4902-a3d8-5948e5d13956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:18.538251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:18.551649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 448/901] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:18.579808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fe8ae-f09f-47cf-bddc-44c4575bed80 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:18.655542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:18.656893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:18.657387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:18.657864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:18.680572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:18.686817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:18.718833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:18.720733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:18.722585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:18.722585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:18.722585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:18.722585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 454/902] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:18.733363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285acd9-26d7-4902-a3d8-5948e5d13956 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:18.833463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:18.926652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:18.933051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285acd9-26d7-4902-a3d8-5948e5d13956 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:18.960482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d327a8-9f86-4291-af00-bcf824a12ec5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:18.998514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:19.004087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:19.004981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:19.005407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:19.005834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:19.006775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:19.006940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:19.008076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:19.008646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:19.009164 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:19.009714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:19.011267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:19.011702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:19.013217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:19.030029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 449/903] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:19.033525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:19.102104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 455/904] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:13:18 2022] POST /identity/v3/services => generated 241 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:13:19.199091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf692c4-b693-41f0-97b5-835d6b67fb26 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:19.249226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018785a-4722-40ac-93fd-b4e7a04f1129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:19.249877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018785a-4722-40ac-93fd-b4e7a04f1129 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:19.250195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018785a-4722-40ac-93fd-b4e7a04f1129 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:19.250467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018785a-4722-40ac-93fd-b4e7a04f1129 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:19.260432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018785a-4722-40ac-93fd-b4e7a04f1129 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:19.262122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018785a-4722-40ac-93fd-b4e7a04f1129 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4018785a-4722-40ac-93fd-b4e7a04f1129 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:19.276173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:19.278020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:19.279877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 450/905] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:13:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:19.284633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:19.456313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403664-615e-4d04-834d-5b80cf302afd None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:19.529394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:19.530969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:19.532035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:19.532782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:19.534541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:19.535444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:19.538674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:19.562874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf323f-abd4-429e-b57c-89cf023aa6e2 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:19.563677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf323f-abd4-429e-b57c-89cf023aa6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:19.563677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf323f-abd4-429e-b57c-89cf023aa6e2 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:19.565090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 451/906] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:19.570838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b06c1-3852-4807-bf04-c7b04b308a7e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:19.571122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b06c1-3852-4807-bf04-c7b04b308a7e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:19.571996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b06c1-3852-4807-bf04-c7b04b308a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:19.582515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 456/907] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 24 14:13:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:19.591332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:19.593585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bf1b06c1-3852-4807-bf04-c7b04b308a7e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:19.594882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bf1b06c1-3852-4807-bf04-c7b04b308a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:19.608028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:19.706734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:19.726728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 452/908] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:19.744816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e09962-e894-4951-b642-da7432f407e3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:19.815669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:19.816336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:19.816685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:19.817014 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:19.913564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:19.916779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:19.986663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:20.062467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 457/909] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:19 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:13:20.067544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b06c1-3852-4807-bf04-c7b04b308a7e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:20.209199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b06c1-3852-4807-bf04-c7b04b308a7e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:20.231218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:20.263658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:20.264344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:20.264643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:20.264982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:20.282163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:20.284157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:20.288408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:20.310324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:20.312108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:20.313752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:20.313752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:20.313752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:20.313752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 453/910] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:20.320476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:20.441679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adb6bf-3b18-4a6a-a06a-3900ded7c9b6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:20.507495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:20.508157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:20.508430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:20.508699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:20.510209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:20.511654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4816a9-985b-4faf-b728-4784d99045a9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:20.511841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:20.525116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:20.526991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 14:13:20.530226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:20.533160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 458/911] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:20.533160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:20.555734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:20.591619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:20.593867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:20.594713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:20.595463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:20.597119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:20.598017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:20.601024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:20.629378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 454/912] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:20.631450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:20.682653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f463248-3d50-456d-bc79-6df201b37aa9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:20.740439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:20.741428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:20.741428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:20.741711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:20.742329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:20.742630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:20.745112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:20.749236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:20.757566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 459/913] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:20.758585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:20.846996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee49b3-3e86-4ec3-835b-c390b19fa47d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:20.894443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a2a7-c2c9-4515-abdb-1bba9258a08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:20.895095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a2a7-c2c9-4515-abdb-1bba9258a08b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:20.895677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:20.896485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a2a7-c2c9-4515-abdb-1bba9258a08b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:20.896894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a2a7-c2c9-4515-abdb-1bba9258a08b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:20.907206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a2a7-c2c9-4515-abdb-1bba9258a08b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:20.908697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a2a7-c2c9-4515-abdb-1bba9258a08b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:20.928081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dd28a-b156-4907-8bef-b5aff58de854 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-24d1a2a7-c2c9-4515-abdb-1bba9258a08b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:20.935232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:20.936996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 455/914] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:20.939577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dae65-38a7-45b0-8f73-edeed02e4b16 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:20.941490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dae65-38a7-45b0-8f73-edeed02e4b16 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:20.941490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dae65-38a7-45b0-8f73-edeed02e4b16 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:20.943815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 456/915] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:20.945333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28dae65-38a7-45b0-8f73-edeed02e4b16 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:20.999724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b939ff-38f3-4cba-9bf4-ce028cd046f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:21.000817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b939ff-38f3-4cba-9bf4-ce028cd046f8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:21.001304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b939ff-38f3-4cba-9bf4-ce028cd046f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:21.001875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b939ff-38f3-4cba-9bf4-ce028cd046f8 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:21.003109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b939ff-38f3-4cba-9bf4-ce028cd046f8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:21.003656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b939ff-38f3-4cba-9bf4-ce028cd046f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:21.006697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b939ff-38f3-4cba-9bf4-ce028cd046f8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:21.021436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 460/916] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 24 14:13:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:21.021436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582cd19-3492-4e36-93ca-c1e362e5d574 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:21.021436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582cd19-3492-4e36-93ca-c1e362e5d574 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:21.021436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582cd19-3492-4e36-93ca-c1e362e5d574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:21.050685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1582cd19-3492-4e36-93ca-c1e362e5d574 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:21.052066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1582cd19-3492-4e36-93ca-c1e362e5d574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:21.150494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28dae65-38a7-45b0-8f73-edeed02e4b16 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:21.175767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:21.190039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 461/917] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:21.190039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582cd19-3492-4e36-93ca-c1e362e5d574 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:21.229835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:21.230993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:21.231289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:21.231620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:21.232527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:21.232889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:21.235508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:21.258257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 457/918] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:21.267262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:21.344938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:21.387107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582cd19-3492-4e36-93ca-c1e362e5d574 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:21.437929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:21.438537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:21.438871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:21.439198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:21.440001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:21.441682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:21.505151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16d23b-8a12-4f86-89ee-c2acc6e65187 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:21.538680 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 462/919] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:21.605926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:21.606649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:21.606929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:21.607202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:21.607986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] RBAC: Adding query filter params (role.id=eb0f55efb2424a3199b8492010e7801c, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=a9c450c65b6849feb8c4250a1ca09fda) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:21.608293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:21.609967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:21.618627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 458/920] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:21 2022] GET /identity/v3/role_assignments?user.id=a9c450c65b6849feb8c4250a1ca09fda&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=eb0f55efb2424a3199b8492010e7801c => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:21.720192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:21.887543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c4302-2bfd-4061-a760-b4d1e985ef0a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:21.934614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:21.943878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:21.944724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:21.945061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:21.945509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:21.946741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:21.949157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:21.965571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:21.970275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:21.971270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 463/921] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 24 14:13:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:21.978037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:22.028182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:22.138463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd418da-250e-475e-b269-a2ef39017403 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:22.188652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c600-8f78-4387-b981-dea582e7c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:22.189381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c600-8f78-4387-b981-dea582e7c810 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:22.189683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c600-8f78-4387-b981-dea582e7c810 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:22.189955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c600-8f78-4387-b981-dea582e7c810 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:22.190727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c600-8f78-4387-b981-dea582e7c810 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:22.191086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c600-8f78-4387-b981-dea582e7c810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:22.192823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c600-8f78-4387-b981-dea582e7c810 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:22.201033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 459/922] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:22.208160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:22.250221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c25c8e-180d-45ec-9b16-6e3d2ea1d4c0 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:22.250596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c25c8e-180d-45ec-9b16-6e3d2ea1d4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:22.250920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c25c8e-180d-45ec-9b16-6e3d2ea1d4c0 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:22.252806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 460/923] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:22.260294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a26c-d7c7-44b4-8362-63e43ab9303a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:22.260738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a26c-d7c7-44b4-8362-63e43ab9303a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:22.261069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a26c-d7c7-44b4-8362-63e43ab9303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:22.287883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9208a26c-d7c7-44b4-8362-63e43ab9303a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:22.289425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9208a26c-d7c7-44b4-8362-63e43ab9303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:22.395199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7852a2-1627-4783-aa16-abb4f81b4801 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:22.398443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 461/924] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:22.465475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:22.466633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:22.467106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:22.467777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:22.468940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:22.469470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:22.471793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:22.482858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 464/925] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:22.488323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a26c-d7c7-44b4-8362-63e43ab9303a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:22.523524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:22.613521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a26c-d7c7-44b4-8362-63e43ab9303a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:22.662768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:22.663535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:22.663878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:22.664176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:22.665704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:22.667466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:22.699634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:22.766622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 462/926] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:22.822347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:22.907400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85355b68-e518-4af0-87e2-65db09d62d09 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:22.993515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:22.994644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:22.995123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:22.995813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:22.996881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:22.999394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:23.018660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:23.020596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:23.022309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 465/927] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:23.036811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:23.225965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:23.252831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fd0a-3029-4a5a-be36-fa5eaaa70110 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:23.317614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:23.318489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:23.318880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:23.319249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:23.320146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:23.320564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:23.323713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:23.331753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 463/928] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:23.342919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:23.517077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a807f91-4ef8-441a-8ac2-87aad7d0daa9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:23.546901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:23.576025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:23.576656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:23.576946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:23.577224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:23.577886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:23.578256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:23.579805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:23.586156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 466/929] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:23.597197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:23.770348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd526083-19e5-4c76-a759-354ce8f77ca7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:23.814759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f90f2-fe03-4ca3-ba86-3997e28e56a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:23.815389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f90f2-fe03-4ca3-ba86-3997e28e56a2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:23.815681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f90f2-fe03-4ca3-ba86-3997e28e56a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:23.815928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f90f2-fe03-4ca3-ba86-3997e28e56a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:23.816649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f90f2-fe03-4ca3-ba86-3997e28e56a2 None admin] RBAC: Adding query filter params (interface=public, service_id=6c165d108a824dec8d7ed821cf814085, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:23.816960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f90f2-fe03-4ca3-ba86-3997e28e56a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:23.818393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f90f2-fe03-4ca3-ba86-3997e28e56a2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:23.826155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 464/930] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:13:23 2022] GET /identity/v3/endpoints?service_id=6c165d108a824dec8d7ed821cf814085&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:23.835841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:24.013067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c3f5-5dd7-46c6-9e4c-0783143c075c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:24.054634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371273-8d00-4107-8a5f-fb463ad2acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:24.055161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371273-8d00-4107-8a5f-fb463ad2acee None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:24.055444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371273-8d00-4107-8a5f-fb463ad2acee None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:24.057050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371273-8d00-4107-8a5f-fb463ad2acee None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:24.057689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c371273-8d00-4107-8a5f-fb463ad2acee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:24.059123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c371273-8d00-4107-8a5f-fb463ad2acee None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:24.066446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 467/931] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:13:23 2022] GET /identity/v3/services => generated 1497 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:13:24.114131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dfa91-ca42-443e-8bab-be32aa4d1d17 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:24.114425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dfa91-ca42-443e-8bab-be32aa4d1d17 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:24.114841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dfa91-ca42-443e-8bab-be32aa4d1d17 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:24.116469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 465/932] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:24.123561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43320836-ed76-4900-bb99-c27252140961 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:24.123851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43320836-ed76-4900-bb99-c27252140961 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:24.124151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43320836-ed76-4900-bb99-c27252140961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:24.145707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-43320836-ed76-4900-bb99-c27252140961 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:24.147134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-43320836-ed76-4900-bb99-c27252140961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:24.203663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:24.246913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 468/933] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:24.314173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371c5f-03df-481c-8e3e-739fd9160690 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:24.314970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371c5f-03df-481c-8e3e-739fd9160690 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:24.315655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371c5f-03df-481c-8e3e-739fd9160690 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:24.318317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 466/934] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:24.325215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829cfe9-8535-4360-bebe-c1f08b7bd8f5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:24.325658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829cfe9-8535-4360-bebe-c1f08b7bd8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:24.325932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829cfe9-8535-4360-bebe-c1f08b7bd8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:24.356357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a829cfe9-8535-4360-bebe-c1f08b7bd8f5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:24.358956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a829cfe9-8535-4360-bebe-c1f08b7bd8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:24.500798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 469/935] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:24.647826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d371c5f-03df-481c-8e3e-739fd9160690 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:24.666400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:24.807062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d371c5f-03df-481c-8e3e-739fd9160690 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:24.834712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829cfe9-8535-4360-bebe-c1f08b7bd8f5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:24.885133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:24.886030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:24.886370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:24.886862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:24.888016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:24.891625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:24.896765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING keystone.server.flask.common [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 14:13:24.964505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.sql.core [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 14:13:24.964505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 14:13:24.964505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [parameters: {'id': '3788114e94944744aa455afb4986dc9f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 14:13:24.964505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81724) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 14:13:24.965202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829cfe9-8535-4360-bebe-c1f08b7bd8f5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:24.968761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3788114e94944744aa455afb4986dc9f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3788114e94944744aa455afb4986dc9f. Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 14:13:24.990351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 14:13:24.991480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application yield resource Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise self.value Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application session.commit() Aug 24 14:13:24.992655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.flush() Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application compat.raise_( Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 14:13:24.994392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 14:13:24.996041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 14:13:24.997005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [parameters: {'id': '3788114e94944744aa455afb4986dc9f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:24.998099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:25.000635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3788114e94944744aa455afb4986dc9f. Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 467/936] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Aug 24 14:13:24 2022] POST /identity/v3/projects => generated 262 bytes in 350 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:25.005513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:25.027065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:25.028405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.028828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.029280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.050910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:25.061626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:25.062866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:25.063946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:25.064773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 470/937] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 24 14:13:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:25.073042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:25.149347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:25.211262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4a674-c3e0-456a-a4d0-b01df66d91b1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:25.263827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f45f0-ee53-430a-a891-6ed06de4b702 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:25.267351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9aca2-6534-4eec-b80b-c0e494adc91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:25.269544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9aca2-6534-4eec-b80b-c0e494adc91a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.270192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9aca2-6534-4eec-b80b-c0e494adc91a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.270736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9aca2-6534-4eec-b80b-c0e494adc91a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.281633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9aca2-6534-4eec-b80b-c0e494adc91a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:25.284615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9aca2-6534-4eec-b80b-c0e494adc91a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b8f9aca2-6534-4eec-b80b-c0e494adc91a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:25.298671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:25.299630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:25.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 468/938] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:25.302053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ee3ef-c70c-4de2-ad49-25224e27e471 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.302497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ee3ef-c70c-4de2-ad49-25224e27e471 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.302952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ee3ef-c70c-4de2-ad49-25224e27e471 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.308481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 469/939] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:25.309597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ee3ef-c70c-4de2-ad49-25224e27e471 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:25.334613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00025cc-e6a2-45de-aea1-4e866b00dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:25.335762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00025cc-e6a2-45de-aea1-4e866b00dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.336257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00025cc-e6a2-45de-aea1-4e866b00dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.337302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00025cc-e6a2-45de-aea1-4e866b00dcc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.338580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00025cc-e6a2-45de-aea1-4e866b00dcc1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:25.339304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00025cc-e6a2-45de-aea1-4e866b00dcc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:25.341957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00025cc-e6a2-45de-aea1-4e866b00dcc1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:25.362684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 471/940] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 24 14:13:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:25.362684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a65a0-ca26-491c-9844-1168adabc131 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.362684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a65a0-ca26-491c-9844-1168adabc131 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.362684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a65a0-ca26-491c-9844-1168adabc131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.385958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e86a65a0-ca26-491c-9844-1168adabc131 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:25.387527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e86a65a0-ca26-491c-9844-1168adabc131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:25.418238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:25.454383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ee3ef-c70c-4de2-ad49-25224e27e471 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:25.475569 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 472/941] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:25.476429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a65a0-ca26-491c-9844-1168adabc131 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:25.535462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:25.551928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:25.553779 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.554622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.556297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.558375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:25.559865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:25.563594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:25.614032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 470/942] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:25.614307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a65a0-ca26-491c-9844-1168adabc131 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:25.679335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:25.681137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.681546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.681947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.704277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:25.707260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:25.737765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:25.739635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:25.741397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:25.741397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:25.741397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:25.741397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 473/943] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:25.748805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:25.780803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:25.800308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:25.936128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78262-c5c6-4565-a4a8-c76f0728b3e4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:25.963247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884a3d9-a9ba-4116-8b35-da7154361021 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:25.986068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:25.986712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:25.986997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:25.987277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:25.988277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:25.988588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:25.990804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:26.018129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 471/944] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:26.020843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:26.030223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:26.031183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.031962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.032369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.033280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:26.035202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:26.052891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:26.054143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:26.055094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:26.056669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 474/945] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:26.063689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:26.103648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:26.169213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:26.175160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d181380-4e47-4960-8f1d-d85ae8bd543d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:26.218017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d3211-7c69-4b09-ae54-07c7b16765b0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:26.229228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bcd3d-ab76-41bd-b47f-f7b5808b09cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:26.230827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bcd3d-ab76-41bd-b47f-f7b5808b09cb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.231442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bcd3d-ab76-41bd-b47f-f7b5808b09cb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.234185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bcd3d-ab76-41bd-b47f-f7b5808b09cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.243526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bcd3d-ab76-41bd-b47f-f7b5808b09cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:26.245129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bcd3d-ab76-41bd-b47f-f7b5808b09cb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-707bcd3d-ab76-41bd-b47f-f7b5808b09cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:26.260233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:26.262235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:26.264725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:26.265795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace2788-3328-4fc9-9806-d8b8fee65e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:26.265795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace2788-3328-4fc9-9806-d8b8fee65e81 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.265795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace2788-3328-4fc9-9806-d8b8fee65e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.267000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace2788-3328-4fc9-9806-d8b8fee65e81 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.267233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 472/946] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:13:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:26.267465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace2788-3328-4fc9-9806-d8b8fee65e81 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:26.267702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace2788-3328-4fc9-9806-d8b8fee65e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:26.268158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2f9ec-a5df-445d-a50b-9fc1c3525a4a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.268429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2f9ec-a5df-445d-a50b-9fc1c3525a4a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.268709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2f9ec-a5df-445d-a50b-9fc1c3525a4a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.269196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace2788-3328-4fc9-9806-d8b8fee65e81 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:26.274866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 473/947] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:26.274866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2f9ec-a5df-445d-a50b-9fc1c3525a4a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:26.286226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 475/948] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:26.286226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f01b24-9a7a-4650-81fc-309a12b42ce2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.286226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f01b24-9a7a-4650-81fc-309a12b42ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.286226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f01b24-9a7a-4650-81fc-309a12b42ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.309167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-64f01b24-9a7a-4650-81fc-309a12b42ce2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:26.310317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-64f01b24-9a7a-4650-81fc-309a12b42ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:26.406744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 476/949] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:26.408668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f01b24-9a7a-4650-81fc-309a12b42ce2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:26.494578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2f9ec-a5df-445d-a50b-9fc1c3525a4a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:26.519802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:26.566528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:26.574717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:26.576188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.576901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.577710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.579500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:26.580353 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:26.583518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:26.630316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 474/950] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 24 14:13:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:26.636973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:26.654540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f01b24-9a7a-4650-81fc-309a12b42ce2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:26.716924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:26.719339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.719816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.720278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.721367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:26.721900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:26.724533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:26.738918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 477/951] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:26.755983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:26.840269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882d65d-f1cf-43ec-8551-1de3f5ec43d4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:26.910437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:26.911606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:26.912080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:26.912550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:26.940194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:26.943169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:26.979560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:26.980848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:26.982126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:26.982126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:26.982126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:26.982126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 475/952] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:26.990610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:26.991759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c048c-257e-4fb1-bbee-ea1a7f3e9272 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:27.027679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:27.070055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:27.082769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:27.084960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:27.085674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:27.086362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:27.088211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:27.092346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:27.117257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:27.118882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:27.120035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:27.120917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 478/953] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:27.126774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:27.161970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ca72-ed23-4b69-a7b5-065beb5dd063 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:27.241496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:27.243456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:27.243970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:27.244442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:27.246344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:27.246872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:27.250642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:27.276593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7d6ca-1f09-414a-8993-5c347741ddc7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:27.283245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 476/954] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:27.291488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:27.329007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:27.329640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:27.329922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:27.330232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:27.331073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:27.331389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:27.332953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:27.345399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 479/955] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:27.346556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:27.421006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:27.459016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:27.469808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55eae9-b4f0-48a8-b339-dc4722eca1fb None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:27.491436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecee7d-d8cd-4b8c-93c7-49cee5480b2f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:27.508332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:27.509505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:27.509783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:27.510277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:27.511015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:27.511230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:27.512938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:27.522578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 480/956] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:27.525827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:27.542496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:27.543706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:27.544343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:27.544960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:27.554801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:27.557615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:27.574476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:27.575768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:27.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:27.585581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 477/957] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:27.587544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:27.646719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b33853-df99-46ce-8339-d46458ca5a8e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:27.682462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:27.687877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:27.688510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:27.688791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:27.689135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:27.689854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:27.691330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:27.750419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc44ec-31d3-4711-b4f1-ad8263d9dcc5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:27.792936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 481/958] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:27.812020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fb62a-3cff-4b5a-8321-890d37f2c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:27.813340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb62a-3cff-4b5a-8321-890d37f2c30d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:27.813973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb62a-3cff-4b5a-8321-890d37f2c30d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:27.814578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb62a-3cff-4b5a-8321-890d37f2c30d None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:27.815759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb62a-3cff-4b5a-8321-890d37f2c30d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:27.816398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb62a-3cff-4b5a-8321-890d37f2c30d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:27.818740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb62a-3cff-4b5a-8321-890d37f2c30d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:27.823236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:27.840644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 478/959] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:27.848594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:28.005580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eba9fb-6d7a-4fc1-ab66-d8310cf12fd2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:28.054925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278ddbb-404e-444a-b3d6-6fcdaad1252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:28.056293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278ddbb-404e-444a-b3d6-6fcdaad1252b None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:28.056872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278ddbb-404e-444a-b3d6-6fcdaad1252b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:28.057428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278ddbb-404e-444a-b3d6-6fcdaad1252b None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/a9c450c65b6849feb8c4250a1ca09fda/roles/eb0f55efb2424a3199b8492010e7801c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:28.060475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278ddbb-404e-444a-b3d6-6fcdaad1252b None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=a9c450c65b6849feb8c4250a1ca09fda, role_id=eb0f55efb2424a3199b8492010e7801c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:28.062623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278ddbb-404e-444a-b3d6-6fcdaad1252b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:28.189929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 482/960] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:13:27 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/a9c450c65b6849feb8c4250a1ca09fda/roles/eb0f55efb2424a3199b8492010e7801c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:13:28.263557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d6ae8-9a86-4063-a522-6f3fed43d444 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:28.264331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d6ae8-9a86-4063-a522-6f3fed43d444 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:28.264331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d6ae8-9a86-4063-a522-6f3fed43d444 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:28.267178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 479/961] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:28.276859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d44bb4-9d95-460d-a21f-c8ecd2b214ba None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:28.277794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d44bb4-9d95-460d-a21f-c8ecd2b214ba None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:28.278548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d44bb4-9d95-460d-a21f-c8ecd2b214ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:28.311674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-09d44bb4-9d95-460d-a21f-c8ecd2b214ba None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:28.316229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-09d44bb4-9d95-460d-a21f-c8ecd2b214ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:28.385305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:28.460796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 483/962] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:28.959150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d6ae8-9a86-4063-a522-6f3fed43d444 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:29.169450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d6ae8-9a86-4063-a522-6f3fed43d444 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:29.239574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:29.241092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:29.241848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:29.243424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:29.244964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:29.248396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:29.342323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.sql.core [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 14:13:29.342323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 14:13:29.342323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [parameters: {'user_id': '926d2f84ec2d47e49ac54d281a8512aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 14:13:29.342323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81724) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 14:13:29.358919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 14:13:29.360630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application yield resource Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise self.value Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application session.commit() Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 14:13:29.362536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.flush() Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application compat.raise_( Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 14:13:29.364262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 14:13:29.369066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 14:13:29.375598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [parameters: {'user_id': '926d2f84ec2d47e49ac54d281a8512aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:29.379475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 14:13:29.382687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 14:13:29.386269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 14:13:29.386269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 14:13:29.386269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 14:13:29.386269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 14:13:29.386269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 14:13:29.386269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:29.386269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 480/963] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 24 14:13:28 2022] POST /identity/v3/users => generated 160 bytes in 420 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:29.387765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d44bb4-9d95-460d-a21f-c8ecd2b214ba None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:29.593517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d44bb4-9d95-460d-a21f-c8ecd2b214ba None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:29.665812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac6e2c-ef2d-4362-bf26-362c114959e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:29.666872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6e2c-ef2d-4362-bf26-362c114959e6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:29.667345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6e2c-ef2d-4362-bf26-362c114959e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:29.667802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6e2c-ef2d-4362-bf26-362c114959e6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:29.681458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac6e2c-ef2d-4362-bf26-362c114959e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:29.683862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac6e2c-ef2d-4362-bf26-362c114959e6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-64ac6e2c-ef2d-4362-bf26-362c114959e6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:29.706381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:29.709570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:29.713890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 484/964] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:13:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:29.720608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:29.841816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:29.890693 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18be6-3238-4872-adcf-3504b81a8174 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:29.890968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18be6-3238-4872-adcf-3504b81a8174 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:29.891107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18be6-3238-4872-adcf-3504b81a8174 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:29.892801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 485/965] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:29.898620 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3623e-b48b-40db-8a53-8127834ab92b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:29.899640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3623e-b48b-40db-8a53-8127834ab92b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:29.900029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3623e-b48b-40db-8a53-8127834ab92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:29.922370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-55c3623e-b48b-40db-8a53-8127834ab92b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:29.923731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55c3623e-b48b-40db-8a53-8127834ab92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:29.931591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc1be-e61a-43ae-9e92-ce51ba1f85e6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:29.959325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:29.992028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:29.993148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:29.993667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:29.994226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:29.995516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:29.995996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:29.998650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:30.042755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 486/966] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:30.045333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e07b9-5a9f-40ed-bbab-70b17acf3b94 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:30.045957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e07b9-5a9f-40ed-bbab-70b17acf3b94 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:30.046665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e07b9-5a9f-40ed-bbab-70b17acf3b94 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:30.055672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 487/967] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:30.059093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677fa0e-9dcd-4266-a702-54361ca32bf1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:30.059779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677fa0e-9dcd-4266-a702-54361ca32bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:30.060364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677fa0e-9dcd-4266-a702-54361ca32bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:30.082664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 481/968] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 24 14:13:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:30.094512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9677fa0e-9dcd-4266-a702-54361ca32bf1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:30.096985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9677fa0e-9dcd-4266-a702-54361ca32bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:30.238285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 488/969] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:30.506948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:30.706506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed97729-55b9-491c-b5a4-e1112e8b48f6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:30.708071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677fa0e-9dcd-4266-a702-54361ca32bf1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:30.737697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:30.769387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:30.770346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:30.770812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:30.771273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:30.772267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:30.774861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:30.789550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:30.791651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:30.792833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 482/970] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:30.800876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:30.889423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:30.912002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677fa0e-9dcd-4266-a702-54361ca32bf1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:30.966444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435a1ce-a6be-4397-bbe1-17f51479a7fd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:30.979073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d10a5-3428-45fe-9871-09da56359427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:30.980001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d10a5-3428-45fe-9871-09da56359427 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:30.980376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d10a5-3428-45fe-9871-09da56359427 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:30.980787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d10a5-3428-45fe-9871-09da56359427 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:30.981685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d10a5-3428-45fe-9871-09da56359427 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:30.984020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d10a5-3428-45fe-9871-09da56359427 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:31.012297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102be989-7cc5-4c9f-b62b-b97d98ec2542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:31.013008 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102be989-7cc5-4c9f-b62b-b97d98ec2542 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:31.013330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102be989-7cc5-4c9f-b62b-b97d98ec2542 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:31.013657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102be989-7cc5-4c9f-b62b-b97d98ec2542 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:31.014448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102be989-7cc5-4c9f-b62b-b97d98ec2542 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:31.014795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102be989-7cc5-4c9f-b62b-b97d98ec2542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:31.016409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102be989-7cc5-4c9f-b62b-b97d98ec2542 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:31.027580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 483/971] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:31.027580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00b23-0771-4edb-96bd-830ef76a1efd None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:31.027580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00b23-0771-4edb-96bd-830ef76a1efd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:31.027580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00b23-0771-4edb-96bd-830ef76a1efd None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:31.027580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 484/972] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:31.029129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd00b23-0771-4edb-96bd-830ef76a1efd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:31.068442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 489/973] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:13:30 2022] POST /identity/v3/services => generated 242 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:13:31.069977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e0b9d-bfe4-4040-9223-4f2d6eda8621 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:31.070350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e0b9d-bfe4-4040-9223-4f2d6eda8621 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:31.070676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e0b9d-bfe4-4040-9223-4f2d6eda8621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:31.094318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:31.094786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-882e0b9d-bfe4-4040-9223-4f2d6eda8621 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:31.096196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-882e0b9d-bfe4-4040-9223-4f2d6eda8621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:31.192598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd00b23-0771-4edb-96bd-830ef76a1efd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:31.199506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 490/974] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:31.245103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:31.245862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:31.246261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:31.246573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:31.247497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:31.247860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:31.249516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:31.262647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 485/975] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:31.265293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e0b9d-bfe4-4040-9223-4f2d6eda8621 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:31.402356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:31.427511 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e0b9d-bfe4-4040-9223-4f2d6eda8621 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:31.466701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:31.467333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:31.467608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:31.467880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:31.468607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] RBAC: Adding query filter params (interface=admin, service_id=6c165d108a824dec8d7ed821cf814085, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:31.468916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:31.470362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:31.476566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 491/976] 10.222.0.49 () {56 vars in 1249 bytes} [Wed Aug 24 14:13:31 2022] GET /identity/v3/endpoints?service_id=6c165d108a824dec8d7ed821cf814085&interface=admin®ion_id=RegionOne => generated 193 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:31.484721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:31.506510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:31.626984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5edffc-d6b8-4da2-a837-c5b6957d28b9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:31.629617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:31.670730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:31.671696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:31.672208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:31.673051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:31.673684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:31.675147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:31.686051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 486/977] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:13:31 2022] GET /identity/v3/services => generated 1727 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:13:31.695176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560a2d5-7313-4aab-b4d0-5f1d1a1544a7 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:31.758178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:31.759841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:31.760303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:31.760730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:31.782369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:31.791717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:31.792763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:31.793619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:31.799901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 492/978] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 24 14:13:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:31.801406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:31.971137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff352d-7acc-41d6-9556-4f17bec55a0b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:32.032869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:32.033795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:32.034246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:32.034674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:32.035734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:32.036192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:32.038467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:32.055354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 487/979] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 24 14:13:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:32.061174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:32.065047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:32.194775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f5230-d801-4721-88a7-0066745bfb5d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:32.236994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:32.237528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:32.237801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:32.238111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:32.250209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:32.251695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:32.267848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:32.268877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:32.269764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:32.269764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:32.269764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:32.274866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 493/980] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:32.277141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:32.315143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:32.439440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79209748-f49c-4e1f-aa82-a745441a9e30 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:32.487235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:32.487844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:32.488179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:32.488507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:32.489310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:32.489676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:32.491430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:32.510198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 488/981] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:32.514873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:32.613027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:32.707184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c45b5-46df-4f61-8703-22af75dcd650 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:32.782234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:32.783221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:32.783717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:32.784201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:32.798623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:32.801474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:32.825141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:32.826340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:32.834051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 494/982] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:13:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:32.834882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:32.988259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:33.001163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1a00-a10a-4653-a127-042ac0d85cee None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:33.044844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4c148-03ad-40f7-8b18-04fd523e8cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:33.045357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4c148-03ad-40f7-8b18-04fd523e8cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:33.045624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4c148-03ad-40f7-8b18-04fd523e8cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:33.045894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4c148-03ad-40f7-8b18-04fd523e8cb0 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:33.046673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4c148-03ad-40f7-8b18-04fd523e8cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:33.046977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4c148-03ad-40f7-8b18-04fd523e8cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:33.048442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4c148-03ad-40f7-8b18-04fd523e8cb0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:33.071709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 489/983] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 24 14:13:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:33.078195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:33.272320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e74f7-e75a-4e1f-939d-19dac9fb62b6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:33.304254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:33.339330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:33.340410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:33.340907 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:33.341626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:33.446341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:33.447906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:33.482055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de454a9-47f0-412a-ab75-84c6b856f619 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:33.482360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de454a9-47f0-412a-ab75-84c6b856f619 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:33.482704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de454a9-47f0-412a-ab75-84c6b856f619 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:33.483970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 490/984] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:33.498036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a03ee3-1a6b-44e3-b66d-9d1c3b75ed07 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:33.498036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a03ee3-1a6b-44e3-b66d-9d1c3b75ed07 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:33.498036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a03ee3-1a6b-44e3-b66d-9d1c3b75ed07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:33.524759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-12a03ee3-1a6b-44e3-b66d-9d1c3b75ed07 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:33.526421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-12a03ee3-1a6b-44e3-b66d-9d1c3b75ed07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:33.609745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 491/985] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:33.625063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 495/986] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:33 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 14:13:33.638251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a03ee3-1a6b-44e3-b66d-9d1c3b75ed07 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:33.732761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:33.767040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a03ee3-1a6b-44e3-b66d-9d1c3b75ed07 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:33.840958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:33.842199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:33.842554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:33.842998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:33.862754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:33.865395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:33.894348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:33.895405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:33.896301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:33.896301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:33.896301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:33.901678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 492/987] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:33.902994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:33.931331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:34.007268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:34.115329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c0b1a-16c8-4612-931f-6948ed8b0712 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:34.158206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114c5ab-6b2c-4358-8d88-6f1a0cc44747 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:34.184068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:34.184895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.185176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.185449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.186471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:34.186471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:34.187877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:34.208350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 496/988] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:34.210410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:34.214500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:34.222582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:34.223632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.224040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.224593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.230619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:34.230619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:34.241132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:34.244176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:34.245858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:34.250230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 493/989] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:34.253666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:34.339170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:34.382365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867497b-7704-4865-b150-2f8d462d6805 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:34.443667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:34.444721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.445220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.446296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.448999 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40721e-87d5-438c-bc3b-c73eee6f847f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:34.458810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:34.462105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:34.491592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:34.493935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:34.496579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 497/990] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:34.499971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:34.500655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:34.501570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.502530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.503235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.504675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:34.505718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:34.510908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:34.520891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 494/991] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:34.531195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:34.637312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:34.662758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129de83-8688-4779-856c-4c90d35cc50c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:34.691663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3964f-7763-4b27-8f4f-800e3408211a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:34.704128 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481e34a-5bed-4483-9db4-9fc97be8eecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:34.705095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e34a-5bed-4483-9db4-9fc97be8eecc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.705552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e34a-5bed-4483-9db4-9fc97be8eecc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.705875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e34a-5bed-4483-9db4-9fc97be8eecc None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.707067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481e34a-5bed-4483-9db4-9fc97be8eecc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:34.707568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481e34a-5bed-4483-9db4-9fc97be8eecc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:34.710040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481e34a-5bed-4483-9db4-9fc97be8eecc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:34.713681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:34.726856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 498/992] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:34.728899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fc233-47b6-4700-8fb1-24a8c1c9e45c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.729218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fc233-47b6-4700-8fb1-24a8c1c9e45c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.729503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fc233-47b6-4700-8fb1-24a8c1c9e45c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.731462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 499/993] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:34.736593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fc233-47b6-4700-8fb1-24a8c1c9e45c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:34.745759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc82c55-f42e-4440-9f96-27edaf1f6589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:34.746433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc82c55-f42e-4440-9f96-27edaf1f6589 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.746720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc82c55-f42e-4440-9f96-27edaf1f6589 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.746989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc82c55-f42e-4440-9f96-27edaf1f6589 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.747635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc82c55-f42e-4440-9f96-27edaf1f6589 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:34.747941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc82c55-f42e-4440-9f96-27edaf1f6589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:34.749358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc82c55-f42e-4440-9f96-27edaf1f6589 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:34.756872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 495/994] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:34.759502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222d19-2cab-40c7-b0e6-1cb16517b82e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.759796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222d19-2cab-40c7-b0e6-1cb16517b82e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.760062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222d19-2cab-40c7-b0e6-1cb16517b82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.780322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fa222d19-2cab-40c7-b0e6-1cb16517b82e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:34.781817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fa222d19-2cab-40c7-b0e6-1cb16517b82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:34.860755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fc233-47b6-4700-8fb1-24a8c1c9e45c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:34.865182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 496/995] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:34.866622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222d19-2cab-40c7-b0e6-1cb16517b82e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:34.925244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46467282-f70f-4316-9de1-443360660600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:34.926216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467282-f70f-4316-9de1-443360660600 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:34.926696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467282-f70f-4316-9de1-443360660600 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:34.927187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467282-f70f-4316-9de1-443360660600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:34.928138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46467282-f70f-4316-9de1-443360660600 None admin] RBAC: Adding query filter params (role.id=eb0f55efb2424a3199b8492010e7801c, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=a9c450c65b6849feb8c4250a1ca09fda) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:34.928574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46467282-f70f-4316-9de1-443360660600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:34.931723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46467282-f70f-4316-9de1-443360660600 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:34.934128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:34.942765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 500/996] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:34 2022] GET /identity/v3/role_assignments?user.id=a9c450c65b6849feb8c4250a1ca09fda&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=eb0f55efb2424a3199b8492010e7801c => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:35.032692 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222d19-2cab-40c7-b0e6-1cb16517b82e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:35.107446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:35.108103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:35.108571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:35.108992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:35.111124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] RBAC: Adding query filter params (interface=public, service_id=f2a4a23b9c4b4691983c4d3ac11af9bd, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:35.111124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:35.113045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:35.122382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 497/997] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:13:34 2022] GET /identity/v3/endpoints?service_id=f2a4a23b9c4b4691983c4d3ac11af9bd&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:35.128225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46467282-f70f-4316-9de1-443360660600 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:35.165815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:35.223662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:35.282934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46467282-f70f-4316-9de1-443360660600 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:35.316237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a4f7-1b94-4a28-84dc-fe8276184399 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:35.327355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:35.327993 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:35.328277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:35.328567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:35.329184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:35.330743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:35.350027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 501/998] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:13:35 2022] GET /identity/v3/services => generated 1727 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:13:35.350373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:35.382900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:35.394247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:35.394247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:35.394247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:35.394247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:35.394247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:35.396755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:35.402845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:35.404789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 498/999] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 24 14:13:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:35.408509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:35.580236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491189f-ad79-4e7b-9133-25aa24095e86 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:35.607458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:35.629194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:35.645400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:35.645400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:35.645400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:35.645400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:35.645400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:35.645400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:35.652426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 502/1000] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:35.657817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:35.810516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd06ef6-5090-4fef-84df-a55a52c7fe2f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:35.857280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f735d9b-6e98-48b2-87dd-d3f35b12055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:35.857914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735d9b-6e98-48b2-87dd-d3f35b12055b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:35.858224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735d9b-6e98-48b2-87dd-d3f35b12055b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:35.858487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735d9b-6e98-48b2-87dd-d3f35b12055b None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:35.859156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735d9b-6e98-48b2-87dd-d3f35b12055b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:35.859447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735d9b-6e98-48b2-87dd-d3f35b12055b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:35.862030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735d9b-6e98-48b2-87dd-d3f35b12055b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:35.867829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 499/1001] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 24 14:13:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:35.874514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:36.067561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807058a9-38fd-402a-a8ce-c11df0369928 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:36.134844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fab1a-7f42-4864-93a5-6b607295cd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:36.135799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fab1a-7f42-4864-93a5-6b607295cd6f None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:36.136218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fab1a-7f42-4864-93a5-6b607295cd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:36.136640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fab1a-7f42-4864-93a5-6b607295cd6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:36.137838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12fab1a-7f42-4864-93a5-6b607295cd6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:36.140705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12fab1a-7f42-4864-93a5-6b607295cd6f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:36.314442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 503/1002] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:37.573846 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab80a8-a709-41aa-991c-6c7f9d781e29 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:37.574699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab80a8-a709-41aa-991c-6c7f9d781e29 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:37.575042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab80a8-a709-41aa-991c-6c7f9d781e29 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:37.577382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 500/1003] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:37.587205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92741-151a-4a99-8f24-9e965ffea98f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:37.588143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92741-151a-4a99-8f24-9e965ffea98f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:37.588143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92741-151a-4a99-8f24-9e965ffea98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:37.621297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bfe92741-151a-4a99-8f24-9e965ffea98f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:37.623350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bfe92741-151a-4a99-8f24-9e965ffea98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:37.665553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72353a-af97-4728-ba71-fb242d13f029 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:37.666102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72353a-af97-4728-ba71-fb242d13f029 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:37.666450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72353a-af97-4728-ba71-fb242d13f029 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:37.668332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 501/1004] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:37.676840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689282ec-de2c-4c0f-bc62-afac8d8e9f1c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:37.677262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689282ec-de2c-4c0f-bc62-afac8d8e9f1c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:37.677703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689282ec-de2c-4c0f-bc62-afac8d8e9f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:37.706715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-689282ec-de2c-4c0f-bc62-afac8d8e9f1c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:37.708164 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:37.708375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-689282ec-de2c-4c0f-bc62-afac8d8e9f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:37.764522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 504/1005] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:37.787611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:37.842626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 502/1006] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:38.238545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe92741-151a-4a99-8f24-9e965ffea98f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:38.284938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689282ec-de2c-4c0f-bc62-afac8d8e9f1c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:38.456139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe92741-151a-4a99-8f24-9e965ffea98f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:38.476716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689282ec-de2c-4c0f-bc62-afac8d8e9f1c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:38.517744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c5002-7420-4efe-93ea-4d3b5a9c09fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:38.518045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7c661-5292-4285-ac2b-b63d27559283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:38.518524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c661-5292-4285-ac2b-b63d27559283 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:38.518786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c5002-7420-4efe-93ea-4d3b5a9c09fb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:38.518896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c661-5292-4285-ac2b-b63d27559283 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:38.519067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7c661-5292-4285-ac2b-b63d27559283 None admin] PATH_INFO: `/v3/services/image` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:38.519287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c5002-7420-4efe-93ea-4d3b5a9c09fb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:38.519662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7c661-5292-4285-ac2b-b63d27559283 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:38.519833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c5002-7420-4efe-93ea-4d3b5a9c09fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:38.520844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c5002-7420-4efe-93ea-4d3b5a9c09fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:38.521020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7c661-5292-4285-ac2b-b63d27559283 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:38.524501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c5002-7420-4efe-93ea-4d3b5a9c09fb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-11d7c661-5292-4285-ac2b-b63d27559283 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:38.531833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:38.532970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:38.534540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 503/1007] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 24 14:13:38 2022] GET /identity/v3/services/image => generated 86 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:38.538336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f5cc5-35bd-416b-a9b9-1d50a26bd648 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:38.538336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f5cc5-35bd-416b-a9b9-1d50a26bd648 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:38.538336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f5cc5-35bd-416b-a9b9-1d50a26bd648 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:38.538336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 504/1008] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:38.541651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f5cc5-35bd-416b-a9b9-1d50a26bd648 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-d20c5002-7420-4efe-93ea-4d3b5a9c09fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:38.542159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:38.543156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 14:13:38.543996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:38.544730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 505/1009] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:38.546521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65766d-0b79-413e-b062-b5ec8e591dd2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:38.547018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65766d-0b79-413e-b062-b5ec8e591dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:38.547518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65766d-0b79-413e-b062-b5ec8e591dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:38.572877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1a65766d-0b79-413e-b062-b5ec8e591dd2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:38.574188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1a65766d-0b79-413e-b062-b5ec8e591dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:38.591007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:38.612752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:38.659477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 506/1010] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:38.662281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65766d-0b79-413e-b062-b5ec8e591dd2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:38.662831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f5cc5-35bd-416b-a9b9-1d50a26bd648 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:38.707328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:38.707946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:38.708203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:38.708548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:38.709189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] RBAC: Adding query filter params (name=image) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:38.709488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:38.710919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:38.716541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 505/1011] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:38.725129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:38.851139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65766d-0b79-413e-b062-b5ec8e591dd2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:38.878329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd21f5-0096-47ea-a95c-4db7e31d3480 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:38.880479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:38.924037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:38.924353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:38.924918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:38.925102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:38.925380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:38.925548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:38.925840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:38.926019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:38.926745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:38.927186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:38.927354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:38.927888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:38.929070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:38.931414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:38.936222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 506/1012] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:38.937233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:38.942057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 507/1013] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:38.951353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:38.996733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:39.106219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c48d2-8280-4f4a-a7ce-fc65dff089bc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:39.151898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2950c-b4a1-46f4-9227-b10ca02d0cf1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:39.169747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:39.170809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:39.171327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:39.171802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:39.172953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:39.173482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:39.176173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:39.186148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 508/1014] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 24 14:13:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:39.192948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:39.212820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:39.213807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:39.214304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:39.214715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:39.217134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:39.219705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:39.226063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 507/1015] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:39.228928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:39.298155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:39.346953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:39.361203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1877-4960-4a2e-9caf-88bf6d8d06c1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:39.411857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d666f5-2023-4d55-8000-be877c8bd475 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:39.432483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234ec7-036e-4359-9193-3de838641e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:39.433591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234ec7-036e-4359-9193-3de838641e54 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:39.434152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234ec7-036e-4359-9193-3de838641e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:39.434674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234ec7-036e-4359-9193-3de838641e54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:39.436275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4234ec7-036e-4359-9193-3de838641e54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:39.439001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4234ec7-036e-4359-9193-3de838641e54 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:39.467055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411c212-47d7-4db0-b1b3-4eaab587949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:39.470618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411c212-47d7-4db0-b1b3-4eaab587949d None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:39.470618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411c212-47d7-4db0-b1b3-4eaab587949d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:39.470618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411c212-47d7-4db0-b1b3-4eaab587949d None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:39.470618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411c212-47d7-4db0-b1b3-4eaab587949d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:39.471475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411c212-47d7-4db0-b1b3-4eaab587949d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:39.560515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 509/1016] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:39.599752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 508/1017] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:13:39 2022] POST /identity/v3/projects => generated 311 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:13:41.002487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4673d-e8d4-41b0-bae6-5bcfecccbb21 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:41.003355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4673d-e8d4-41b0-bae6-5bcfecccbb21 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:41.003355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4673d-e8d4-41b0-bae6-5bcfecccbb21 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:41.004674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 510/1018] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:41.011644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c18e-a2f5-4777-aa1e-008cf58a5e80 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:41.012162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c18e-a2f5-4777-aa1e-008cf58a5e80 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:41.012768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c18e-a2f5-4777-aa1e-008cf58a5e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:41.046551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-10b2c18e-a2f5-4777-aa1e-008cf58a5e80 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:41.048591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-10b2c18e-a2f5-4777-aa1e-008cf58a5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:41.202709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 509/1019] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:41.735853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4673d-e8d4-41b0-bae6-5bcfecccbb21 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:41.755917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67ee7c-b425-4f91-9ce7-fc40829babd1 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:41.756424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67ee7c-b425-4f91-9ce7-fc40829babd1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:41.756926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67ee7c-b425-4f91-9ce7-fc40829babd1 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:41.760277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 510/1020] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:41.769749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94d97-d679-41a9-b9ef-5bec5bb5e6d9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:41.770285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94d97-d679-41a9-b9ef-5bec5bb5e6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:41.770779 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94d97-d679-41a9-b9ef-5bec5bb5e6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:41.807145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-01d94d97-d679-41a9-b9ef-5bec5bb5e6d9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:41.809399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-01d94d97-d679-41a9-b9ef-5bec5bb5e6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:41.825548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:41.936441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4673d-e8d4-41b0-bae6-5bcfecccbb21 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:41.938617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 511/1021] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:42.009756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc333e-b534-4859-b5c6-b26c237e32c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:42.010754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc333e-b534-4859-b5c6-b26c237e32c6 None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:42.011262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc333e-b534-4859-b5c6-b26c237e32c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:42.011766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc333e-b534-4859-b5c6-b26c237e32c6 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:42.013066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc333e-b534-4859-b5c6-b26c237e32c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:42.015879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc333e-b534-4859-b5c6-b26c237e32c6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:42.103835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 511/1022] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:13:41 2022] POST /identity/v3/services => generated 242 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:13:42.222051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbe5c-41b6-4c6d-9db4-385c76d6cc1c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:42.222051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbe5c-41b6-4c6d-9db4-385c76d6cc1c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:42.222051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbe5c-41b6-4c6d-9db4-385c76d6cc1c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:42.222851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 512/1023] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:42.229499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4d683-34f2-43d5-ab02-65e2ad82c3a9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:42.229901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4d683-34f2-43d5-ab02-65e2ad82c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:42.230159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4d683-34f2-43d5-ab02-65e2ad82c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:42.250186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a6c4d683-34f2-43d5-ab02-65e2ad82c3a9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:42.251464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a6c4d683-34f2-43d5-ab02-65e2ad82c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:42.255027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:42.270816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bbe5c-41b6-4c6d-9db4-385c76d6cc1c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:42.373036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 512/1024] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:42.456646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bbe5c-41b6-4c6d-9db4-385c76d6cc1c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:42.519949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:42.521086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:42.521510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:42.521961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:42.524622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:42.527673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:42.531319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 513/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:42.541349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4d683-34f2-43d5-ab02-65e2ad82c3a9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:42.649240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:42.667762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:42.730190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4d683-34f2-43d5-ab02-65e2ad82c3a9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:42.793090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:42.793302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ef62-7e98-419f-a3bc-fb1e091d7b24 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:42.793984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:42.794454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:42.794864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:42.796082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:42.798518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:42.813706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:42.844735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:42.845496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:42.845775 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:42.846083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:42.858543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:42.863656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:42.870696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:42.871556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 514/1026] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 24 14:13:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:42.873613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:42.929912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 513/1027] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:13:42 2022] POST /identity/v3/users => generated 285 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:13:42.978140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:43.026034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d8338-03f5-4e67-ae2a-3033057dfb93 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:43.072700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:43.073246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:43.073536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:43.073789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:43.074552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:43.074852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:43.076345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:43.087407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 515/1028] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 24 14:13:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:43.093716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:43.256442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1018e-30e2-4e0d-8597-4456dcb9e40f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:43.303079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:43.303079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:43.303079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:43.303079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:43.316146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:43.317649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:43.334421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:43.335728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:43.337000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:43.337000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:43.337000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:43.337000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 514/1029] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:43.343893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:43.470632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05053a7a-1991-4090-828f-322981f022c1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:43.517773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:43.527714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:43.528798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:43.529293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:43.529772 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:43.531280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:43.531817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:43.534094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:43.549756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 516/1030] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:43.556516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:43.613420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:43.672119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c131482-1b1a-4e9d-a6ab-77c015b7e42e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:43.708222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:43.708953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:43.709226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:43.709498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:43.718588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:43.720233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:43.734873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:43.735921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:43.736894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:43.740201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 515/1031] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:13:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:43.741903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:43.886106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad712b2-3677-44b0-878c-56af56325218 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:43.958810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d1eaa-d35d-494b-bd95-d66b48730c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:43.959841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1eaa-d35d-494b-bd95-d66b48730c72 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:43.960323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1eaa-d35d-494b-bd95-d66b48730c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:43.960852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1eaa-d35d-494b-bd95-d66b48730c72 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:43.962839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1eaa-d35d-494b-bd95-d66b48730c72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:43.963363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1eaa-d35d-494b-bd95-d66b48730c72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:43.966111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1eaa-d35d-494b-bd95-d66b48730c72 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:44.000739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 517/1032] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:13:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:44.010612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:44.205568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430207e-d37a-410d-a64a-7de30b511d81 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:44.209899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:44.263437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:44.264070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:44.264336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:44.264610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:44.353292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:44.354834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:44.455511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d2f8-0604-45ec-9961-c4512a441f3e None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:44.456413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d2f8-0604-45ec-9961-c4512a441f3e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:44.456413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d2f8-0604-45ec-9961-c4512a441f3e None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:44.457652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 518/1033] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:44.463408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7413b4-ead9-4aeb-b44d-20ea24633dcd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:44.463713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7413b4-ead9-4aeb-b44d-20ea24633dcd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:44.463948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7413b4-ead9-4aeb-b44d-20ea24633dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:44.483513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8b7413b4-ead9-4aeb-b44d-20ea24633dcd None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:44.484784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8b7413b4-ead9-4aeb-b44d-20ea24633dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:44.497911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 516/1034] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:13:44.508198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:44.514419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:44.549272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:44.577669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 519/1035] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:44.657060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425b376-6d40-4383-b4aa-b3f7a9eb6eef None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:44.702901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:44.703627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:44.703885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:44.704173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:44.718494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:44.720151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:44.739989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:44.741916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:44.748869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:44.748869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:44.748869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:44.748869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 517/1036] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:44.749613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7413b4-ead9-4aeb-b44d-20ea24633dcd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:44.857768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:44.936829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7413b4-ead9-4aeb-b44d-20ea24633dcd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:44.946481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:44.985290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:44.986007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:44.986340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:44.986598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:44.987342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:44.987510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:44.988980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.004587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 520/1037] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:45.011205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:45.027667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305eaa04-7af1-49e8-b010-18333bad4340 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.064843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:45.097444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:45.098399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.098756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.099110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] PATH_INFO: `/v3/services/image` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.099722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:45.101209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:45.111279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:45.113301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:45.115556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:45.119004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 518/1038] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 24 14:13:44 2022] GET /identity/v3/services/image => generated 86 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:45.119815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:45.153526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d80953-bc14-424b-b283-2ea7aae92eb1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.212112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:45.213233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.213684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.214158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.225854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:45.229001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:45.249790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:45.250855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:45.251800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:45.254341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 521/1039] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:45.260217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:45.277739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9beef0-6aa7-46b0-ae70-bc36eedd311d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.302403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:45.318994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:45.319623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.319913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.320735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.321417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] RBAC: Adding query filter params (name=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:45.321719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:45.323212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.328625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 519/1040] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:45.335067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:45.393418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:45.396687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909c78d-b0bf-4b30-8859-767baebf01e6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.440023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:45.440659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.440979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.441224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.442045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:45.442369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:45.444456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.460083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 522/1041] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:45.466903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:45.475419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020bcfa-0f79-4981-8985-f24ffbb7e615 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.521474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e09ec-4ce3-4a3a-b2a6-f498ec4af128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:45.522371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e09ec-4ce3-4a3a-b2a6-f498ec4af128 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.522857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e09ec-4ce3-4a3a-b2a6-f498ec4af128 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.523333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e09ec-4ce3-4a3a-b2a6-f498ec4af128 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.525044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e09ec-4ce3-4a3a-b2a6-f498ec4af128 None admin] RBAC: Adding query filter params (type=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:45.525584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e09ec-4ce3-4a3a-b2a6-f498ec4af128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:45.527870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e09ec-4ce3-4a3a-b2a6-f498ec4af128 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.536675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 520/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:45.538714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f91b0-4e0b-4ec1-86d6-7a7ec6e8e20a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.539255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f91b0-4e0b-4ec1-86d6-7a7ec6e8e20a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.539667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f91b0-4e0b-4ec1-86d6-7a7ec6e8e20a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.542424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 521/1043] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:45.544904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f91b0-4e0b-4ec1-86d6-7a7ec6e8e20a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:45.595883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20dcb-6e7f-4156-8e2f-14aeb2fb0145 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.626958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:45.643448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94b699-57a8-44cb-986d-8613eb317458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:45.644518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94b699-57a8-44cb-986d-8613eb317458 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.646734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94b699-57a8-44cb-986d-8613eb317458 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.646734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94b699-57a8-44cb-986d-8613eb317458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.647025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94b699-57a8-44cb-986d-8613eb317458 None admin] RBAC: Adding query filter params (role.id=eb0f55efb2424a3199b8492010e7801c, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=613e985e94c54913a5e6b93367157fc0) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:45.647449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94b699-57a8-44cb-986d-8613eb317458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:45.650703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94b699-57a8-44cb-986d-8613eb317458 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.660886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 523/1044] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:45 2022] GET /identity/v3/role_assignments?user.id=613e985e94c54913a5e6b93367157fc0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=eb0f55efb2424a3199b8492010e7801c => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:45.663205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b30a4-4df3-40f3-af33-134ee7fba024 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.663722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b30a4-4df3-40f3-af33-134ee7fba024 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.664303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b30a4-4df3-40f3-af33-134ee7fba024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.693282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d99b30a4-4df3-40f3-af33-134ee7fba024 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:45.694966 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d99b30a4-4df3-40f3-af33-134ee7fba024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:45.708397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f91b0-4e0b-4ec1-86d6-7a7ec6e8e20a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.729748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:45.775345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:45.776087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:45.776501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:45.776910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:45.778323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] RBAC: Adding query filter params (interface=public, service_id=dd0b40e2e2074a72a2634d1444ad104b, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:45.778736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:45.780732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:45.793818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 522/1045] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:13:45 2022] GET /identity/v3/endpoints?service_id=dd0b40e2e2074a72a2634d1444ad104b&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:45.796054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:45.834623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 524/1046] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:45.955687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255f88b-1bed-41ad-a1da-96c62b021ea3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:45.973793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:46.000214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:46.000846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:46.001279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:46.001438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:46.002098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:46.003540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:46.012612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 523/1047] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:13:45 2022] GET /identity/v3/services => generated 1957 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:13:46.149284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b30a4-4df3-40f3-af33-134ee7fba024 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:46.323629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b30a4-4df3-40f3-af33-134ee7fba024 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:46.397564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:46.399408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:46.399765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:46.400214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:46.422556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:46.432416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:46.434545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:46.436196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 525/1048] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:46.447612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:46.643762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e155d-c078-4ae0-be69-7951720e05be None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:46.693684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:46.694475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:46.694855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:46.695242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:46.696793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:46.697102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:46.698561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:46.709593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 524/1049] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:46.716894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:46.749919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:46.920136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0808d98-1843-44d1-b139-372cf3ee7e5e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:46.986347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:46.987815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:46.988517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:46.989166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] PATH_INFO: `/v3/users/d04ebf64b5724be5bb236e42c18b701c` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:47.015924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] RBAC: Authorizing `identity:get_user(user_id=d04ebf64b5724be5bb236e42c18b701c)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:47.018959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:47.022898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 526/1050] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:13:46 2022] GET /identity/v3/users/d04ebf64b5724be5bb236e42c18b701c => generated 285 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:47.029714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:47.086551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:47.164815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945eefc-24b3-4e66-bae6-d2151a9818e9 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:47.208918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:47.209547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:47.209825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:47.210134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:47.297473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:47.298920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:47.350330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:47.468422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 525/1051] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:47 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:13:47.479320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:47.575403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:47.616411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db2af5-bd4d-471e-a4ec-a895e9f824e0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:47.667358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7727c-eeb6-4af5-add0-ee8cb9186555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:47.668385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7727c-eeb6-4af5-add0-ee8cb9186555 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:47.668870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7727c-eeb6-4af5-add0-ee8cb9186555 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:47.669396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7727c-eeb6-4af5-add0-ee8cb9186555 None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:47.671099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7727c-eeb6-4af5-add0-ee8cb9186555 None admin] RBAC: Authorizing `identity:get_project(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:47.672924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7727c-eeb6-4af5-add0-ee8cb9186555 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:47.681632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 527/1052] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:13:47 2022] GET /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f => generated 311 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:47.683750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:47.827256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e07ec-b2f2-4a86-9aed-eb7704cca90e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:47.872280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318c33b-6d8e-4b89-82bb-bdd2b1ff3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:47.872923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318c33b-6d8e-4b89-82bb-bdd2b1ff3076 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:47.873201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318c33b-6d8e-4b89-82bb-bdd2b1ff3076 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:47.873465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318c33b-6d8e-4b89-82bb-bdd2b1ff3076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:47.874258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318c33b-6d8e-4b89-82bb-bdd2b1ff3076 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f, user.id=d04ebf64b5724be5bb236e42c18b701c) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:47.874571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318c33b-6d8e-4b89-82bb-bdd2b1ff3076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:47.876190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318c33b-6d8e-4b89-82bb-bdd2b1ff3076 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:47.884219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 526/1053] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:47 2022] GET /identity/v3/role_assignments?user.id=d04ebf64b5724be5bb236e42c18b701c&scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:47.889332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664307d-db0e-4c70-a67b-2f6bfda70a02 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:47.889888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664307d-db0e-4c70-a67b-2f6bfda70a02 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:47.890544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664307d-db0e-4c70-a67b-2f6bfda70a02 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:47.893707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 528/1054] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:47.902952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dde7-25c8-493e-b71f-278ef4b3b8bf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:47.903238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dde7-25c8-493e-b71f-278ef4b3b8bf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:47.903567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dde7-25c8-493e-b71f-278ef4b3b8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:47.911320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:47.931223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c663dde7-25c8-493e-b71f-278ef4b3b8bf None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:47.933135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c663dde7-25c8-493e-b71f-278ef4b3b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:48.031747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 527/1055] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:48.317180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664307d-db0e-4c70-a67b-2f6bfda70a02 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:48.463091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664307d-db0e-4c70-a67b-2f6bfda70a02 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:48.506139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:48.506952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:48.507364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:48.508285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:48.518796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:48.524026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:48.529512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:48.531640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:48.532918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 529/1056] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 24 14:13:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:48.538970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663dde7-25c8-493e-b71f-278ef4b3b8bf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:48.728137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663dde7-25c8-493e-b71f-278ef4b3b8bf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:48.785410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:48.794655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:48.795761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:48.811004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:48.811004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:48.811004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:48.811004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:48.811004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:48.822573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 528/1057] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 24 14:13:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:48.824946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:48.953948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8155a9c-8351-4c48-b654-58c13c4fe44f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:48.997318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23136cc-a757-4892-8306-085e764b74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:48.998102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23136cc-a757-4892-8306-085e764b74fe None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:48.998527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23136cc-a757-4892-8306-085e764b74fe None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:48.998955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23136cc-a757-4892-8306-085e764b74fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.012330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23136cc-a757-4892-8306-085e764b74fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:49.014035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23136cc-a757-4892-8306-085e764b74fe None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b23136cc-a757-4892-8306-085e764b74fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:49.033869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:49.035991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:49.037134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:49.037134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:49.037134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:49.037134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 530/1058] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:49.043696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:49.084363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532b92-50d6-4f70-b473-000234df8dab None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:49.084708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532b92-50d6-4f70-b473-000234df8dab None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:49.084960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f532b92-50d6-4f70-b473-000234df8dab None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.087023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 531/1059] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:49.092589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8f582-4cd2-4e7c-81f9-1ff2a2760349 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:49.092915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8f582-4cd2-4e7c-81f9-1ff2a2760349 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:49.093136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8f582-4cd2-4e7c-81f9-1ff2a2760349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.113089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-69d8f582-4cd2-4e7c-81f9-1ff2a2760349 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:49.114342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-69d8f582-4cd2-4e7c-81f9-1ff2a2760349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:49.171353 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:49.196431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 532/1060] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:49.231041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6813a-d21e-498c-8047-15a4b2d0313d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:49.297149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:49.298160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:49.298654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:49.299144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.300351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:49.300892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:49.303588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:49.338048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 529/1061] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:49.341423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8f582-4cd2-4e7c-81f9-1ff2a2760349 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:49.471741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:49.549925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8f582-4cd2-4e7c-81f9-1ff2a2760349 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:49.556357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:49.618458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:49.619308 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:49.619723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:49.620212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.634788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:49.636609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93b700-65df-4a76-aba2-29ba7d12edb6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:49.636899 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:49.662349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:49.664614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:49.665668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:49.666709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 533/1062] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:13:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:49.672992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:49.697206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:49.698931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:49.699678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:49.700214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.701268 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:49.704142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:49.722728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:49.724795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:49.728059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 530/1063] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 24 14:13:49 2022] GET /identity/v3/services/image => generated 86 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:49.732924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:49.738669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:49.835323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064858b6-933e-4d82-8555-4c354c85060d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:49.878742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110d55d-1762-4297-8161-4508d246f9e8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:49.896873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:49.897935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:49.899469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:49.899873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.900534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:49.901133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:49.901686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:49.904560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:49.922923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:49.923555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:49.923891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:49.924173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:49.924855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:49.925153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:49.926726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:49.938414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 531/1064] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:49.939492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 534/1065] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:13:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:49.940882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:49.945200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:50.095404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec88361-64c7-473b-a14c-501eaace7ce2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:50.136089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:50.141814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fa80a-fb95-4446-a512-a3142a8f521c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:50.162563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:50.163560 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.164545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.165006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.166139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:50.166653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:50.168970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:50.180432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 532/1066] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:50.190361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:50.206446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:50.207457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.207876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.208320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.222945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:50.224961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:50.247783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:50.259120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:50.261151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:50.261151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:50.261151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:50.261151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 535/1067] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:50.261151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:50.299603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:50.394081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f03fe-ba35-4ed0-bf55-9dd974b850d4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:50.415954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fc8cb-1856-4f01-b409-187025c65633 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:50.437373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30717bca-b701-4ee7-a591-bf854fc8530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:50.438023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30717bca-b701-4ee7-a591-bf854fc8530c None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.438315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30717bca-b701-4ee7-a591-bf854fc8530c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.438591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30717bca-b701-4ee7-a591-bf854fc8530c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.439279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30717bca-b701-4ee7-a591-bf854fc8530c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:50.440747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30717bca-b701-4ee7-a591-bf854fc8530c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:50.465588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a85d-978a-4d6c-a1db-174bb3b95253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:50.466314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a85d-978a-4d6c-a1db-174bb3b95253 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.466760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a85d-978a-4d6c-a1db-174bb3b95253 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.466927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a85d-978a-4d6c-a1db-174bb3b95253 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.468299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a85d-978a-4d6c-a1db-174bb3b95253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:50.468609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a85d-978a-4d6c-a1db-174bb3b95253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:50.470016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a85d-978a-4d6c-a1db-174bb3b95253 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:50.490368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 536/1068] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:50.490368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a8f0-6732-4dde-8cf9-1808e32d10d4 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.490368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a8f0-6732-4dde-8cf9-1808e32d10d4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.490368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a8f0-6732-4dde-8cf9-1808e32d10d4 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.490368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 537/1069] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:50.492520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551a8f0-6732-4dde-8cf9-1808e32d10d4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:50.508958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:50.536890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 533/1070] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:13:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:13:50.538958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ff550-d631-44cf-9c73-d1798afd444e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.539224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ff550-d631-44cf-9c73-d1798afd444e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.539660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ff550-d631-44cf-9c73-d1798afd444e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.562685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6f9ff550-d631-44cf-9c73-d1798afd444e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:50.563948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6f9ff550-d631-44cf-9c73-d1798afd444e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:50.651562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551a8f0-6732-4dde-8cf9-1808e32d10d4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:50.658861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 534/1071] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:50.697701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:50.698599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.699014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.699447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.708054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:50.709923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:50.723136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:50.724437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:50.727720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 538/1072] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:50.731357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ff550-d631-44cf-9c73-d1798afd444e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:50.777592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:50.880491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ff550-d631-44cf-9c73-d1798afd444e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:50.924434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:50.925123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:50.925496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:50.925863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:50.926726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:50.927204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:50.928869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:50.938548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:50.958034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 535/1073] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:50.962966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:51.055145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:51.121020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6c961-6a4e-4e49-9b23-086473c8d86c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:51.138475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:51.155655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d2d8b-d63e-4db3-a01b-7f85aa3d77e1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:51.175693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:51.176631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:51.177071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:51.177536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:51.198417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:51.208574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:51.210727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:51.212189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:51.212943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 539/1074] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:51.220575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:51.227318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:51.228396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:51.228868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:51.229336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/613e985e94c54913a5e6b93367157fc0/roles/eb0f55efb2424a3199b8492010e7801c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:51.233252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=613e985e94c54913a5e6b93367157fc0, role_id=eb0f55efb2424a3199b8492010e7801c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:51.235903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:51.342260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 536/1075] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:13:50 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/613e985e94c54913a5e6b93367157fc0/roles/eb0f55efb2424a3199b8492010e7801c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:13:51.390754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f38d4-4178-4e85-9733-6203175ac369 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:51.446493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:51.462257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:51.463425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:51.463979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:51.464526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:51.465897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:51.466532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:51.469243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:51.485666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 540/1076] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:51.492329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:51.639981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cf5c-d697-418c-b987-1230c5a90b5e None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:51.703864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:51.704850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:51.705367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:51.705880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] PATH_INFO: `/v3/users/d04ebf64b5724be5bb236e42c18b701c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:51.708632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] RBAC: Authorizing `identity:get_user(user_id=d04ebf64b5724be5bb236e42c18b701c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:51.711444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:51.719351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 537/1077] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:13:51 2022] GET /identity/v3/users/d04ebf64b5724be5bb236e42c18b701c => generated 285 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:51.721819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:51.928599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb83710-4e56-4b04-8ca9-0e1fa27f3e29 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:52.002851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e022c-17ca-43e5-972d-cf43e7147bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:52.003876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e022c-17ca-43e5-972d-cf43e7147bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:52.004342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e022c-17ca-43e5-972d-cf43e7147bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:52.004830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e022c-17ca-43e5-972d-cf43e7147bd1 None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:52.007062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e022c-17ca-43e5-972d-cf43e7147bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:52.010645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e022c-17ca-43e5-972d-cf43e7147bd1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:52.014060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 541/1078] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:13:51 2022] GET /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f => generated 311 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:52.024527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:52.070628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:52.236083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87475f86-0dd2-49d3-b8c0-fd0ceed6785f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:52.303434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacebe8-5088-4fd6-a5cf-44e0ddef7a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:52.304746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacebe8-5088-4fd6-a5cf-44e0ddef7a40 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:52.305172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacebe8-5088-4fd6-a5cf-44e0ddef7a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:52.305493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacebe8-5088-4fd6-a5cf-44e0ddef7a40 None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f/users/d04ebf64b5724be5bb236e42c18b701c/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:52.307984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacebe8-5088-4fd6-a5cf-44e0ddef7a40 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f, user_id=d04ebf64b5724be5bb236e42c18b701c, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:52.309621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacebe8-5088-4fd6-a5cf-44e0ddef7a40 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:52.403754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 538/1079] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:13:52 2022] PUT /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f/users/d04ebf64b5724be5bb236e42c18b701c/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:13:52.742400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de308117-106e-4335-967e-506a781de7c6 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:52.742955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de308117-106e-4335-967e-506a781de7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:52.743390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de308117-106e-4335-967e-506a781de7c6 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:52.746020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 542/1080] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:52.754501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa546c8-3044-4ad1-ad53-d5a562751695 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:52.755276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa546c8-3044-4ad1-ad53-d5a562751695 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:52.755276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa546c8-3044-4ad1-ad53-d5a562751695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:52.776972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6aa546c8-3044-4ad1-ad53-d5a562751695 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:52.778292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6aa546c8-3044-4ad1-ad53-d5a562751695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:52.871185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 539/1081] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:53.147585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de308117-106e-4335-967e-506a781de7c6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:53.214452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:53.261058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de308117-106e-4335-967e-506a781de7c6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:53.307293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f638910-9511-446e-895b-21ceaab74fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:53.308116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f638910-9511-446e-895b-21ceaab74fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:53.308518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f638910-9511-446e-895b-21ceaab74fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:53.308915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f638910-9511-446e-895b-21ceaab74fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:53.403971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f638910-9511-446e-895b-21ceaab74fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:53.405842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f638910-9511-446e-895b-21ceaab74fd9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:53.480426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:53.535845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 543/1082] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:13:53.542826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa546c8-3044-4ad1-ad53-d5a562751695 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:53.564166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:53.604822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480fc3d-76df-4d00-91a3-3b1f28cba042 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:53.605297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480fc3d-76df-4d00-91a3-3b1f28cba042 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:53.605448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480fc3d-76df-4d00-91a3-3b1f28cba042 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:53.606999 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 544/1083] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:13:53.612912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187ae2a-98b4-4aa3-8558-635e1728b088 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:53.613173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187ae2a-98b4-4aa3-8558-635e1728b088 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:53.613444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187ae2a-98b4-4aa3-8558-635e1728b088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:53.634228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b187ae2a-98b4-4aa3-8558-635e1728b088 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:53.635526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b187ae2a-98b4-4aa3-8558-635e1728b088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:53.727841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 545/1084] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:53.742986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa546c8-3044-4ad1-ad53-d5a562751695 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:53.807559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:53.808595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:53.808966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:53.809439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:53.829069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:53.832434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:53.862200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:53.863412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:53.864771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:53.864771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:53.864771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:53.866456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 540/1085] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:53.874027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187ae2a-98b4-4aa3-8558-635e1728b088 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:54.003792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:54.015266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:54.031357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187ae2a-98b4-4aa3-8558-635e1728b088 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:54.078396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:54.079027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:54.079314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:54.079572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:54.080231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:54.080539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:54.082798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:54.110383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 546/1086] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:54.122198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:54.191413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45521bab-76f8-41b8-9899-ff42db40243d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:54.233395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815093c-f4b1-4e32-88ce-b1fb9785c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:54.234092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815093c-f4b1-4e32-88ce-b1fb9785c24d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:54.234412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815093c-f4b1-4e32-88ce-b1fb9785c24d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:54.234632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815093c-f4b1-4e32-88ce-b1fb9785c24d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:54.248209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815093c-f4b1-4e32-88ce-b1fb9785c24d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-c815093c-f4b1-4e32-88ce-b1fb9785c24d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:54.252998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:54.255349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 14:13:54.257201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:54.260021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 541/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 24 14:13:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:54.262178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:54.470566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0718a8-27bf-44e3-b739-ace8bb6ec481 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:54.516017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:54.516756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:54.517220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:54.518007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:54.518827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:54.519269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:54.520815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:54.530689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b495892-d0b8-4b69-92e2-3c807a9c4544 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:54.530968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b495892-d0b8-4b69-92e2-3c807a9c4544 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:54.531241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b495892-d0b8-4b69-92e2-3c807a9c4544 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:54.531954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 547/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 24 14:13:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:54.532557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 542/1089] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:54.537667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:54.538735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5acc-2730-4c76-9213-dcd84b00c0ef None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:54.539211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5acc-2730-4c76-9213-dcd84b00c0ef None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:54.539860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5acc-2730-4c76-9213-dcd84b00c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:54.553258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:54.561398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8bbe5acc-2730-4c76-9213-dcd84b00c0ef None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:54.562648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8bbe5acc-2730-4c76-9213-dcd84b00c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:54.658963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 543/1090] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:13:54.683977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c5d49-c3f8-4df4-b100-1f9e7b23ea66 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:54.724929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:54.725723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:54.726781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:54.727149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:54.740117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:54.742367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:54.759757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:54.761177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:54.762679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:54.762679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:54.762679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:54.762679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 548/1091] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:54.768033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe5acc-2730-4c76-9213-dcd84b00c0ef None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:54.869655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:54.936534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:54.953119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe5acc-2730-4c76-9213-dcd84b00c0ef None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:54.992740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:54.993455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:54.993760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:54.994031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:54.994693 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:54.994995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:54.996456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:55.010403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 544/1092] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:55.016524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.020140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:55.066722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bcb70-4e41-4559-9921-03d80f17cb1f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:55.107305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:55.107950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:55.108329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:55.108496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:55.121382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:13:55.126812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:55.127854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:13:55.128727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 549/1093] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:13:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:13:55.134240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.162891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73fe3e-c56e-4d05-9fa4-6479dfa089de None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:55.178624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:55.204570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:55.205220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:55.205742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:55.206048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:55.216294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:55.217940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:13:55.230554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:55.231675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:55.232641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 545/1094] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:55.238434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.255661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba022b2d-1082-4037-88f9-a8426d7a9856 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:55.289332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:55.305331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:55.305957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:55.306273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:55.306538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:55.307214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:55.307511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:55.308905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:55.318504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 550/1095] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:55.326953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.365937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e429c2f-3e00-4d76-b7be-09f6e6dc2c9c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:55.407905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:55.408956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:55.409452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:55.409929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:55.411387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:55.411940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:55.415267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:55.455530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 546/1096] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:55.455818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024222-c552-4315-9611-da6a4509f200 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:55.457481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.494959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:55.496126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:55.496126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:55.496420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] PATH_INFO: `/v3/users/d04ebf64b5724be5bb236e42c18b701c` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:55.497693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] RBAC: Authorizing `identity:get_user(user_id=d04ebf64b5724be5bb236e42c18b701c)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:55.499461 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:55.501224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 551/1097] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/users/d04ebf64b5724be5bb236e42c18b701c => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:55.502688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:55.508107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.589825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575404a-4857-4cad-a6eb-d10be7ff3053 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:55.593321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:55.630177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:55.630797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:55.631074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:55.631347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:55.659837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d5650-d627-42db-b4ed-44b54b43600f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:55.703817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:55.704790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:55.705066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:55.705331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:55.718330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:55.719830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:55.754375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:55.804970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:55.806995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:55.846950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 547/1098] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:13:55.853447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.865593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:55.922493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 552/1099] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 14:13:55.929075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:55.990506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e4683-ec78-41d5-abf9-4e56e4a0df55 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:56.026230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:56.033183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:56.033898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:56.034209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:56.044915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:56.047866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:56.049791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:56.059198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4d805-7a0c-482c-ad85-53c8136e9ceb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:13:56.067555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:56.069709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:13:56.071304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:13:56.071304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:13:56.071304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:56.071304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 548/1100] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:56.075629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:56.112638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:56.113464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:56.113834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:56.114207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:56.115684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] RBAC: Authorizing `identity:get_project(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:56.117611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:56.120112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 553/1101] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:13:55 2022] GET /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:56.126483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:56.150361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:56.194607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d872d-943b-4042-a665-5f34ef833c8f None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:56.234928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:56.235537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:56.235814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:56.236082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:56.236727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:56.237030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:56.238460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:56.247299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ed563-6000-4e6f-a9b8-6786d943481d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:56.251053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 549/1102] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:56.257276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:56.288000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:56.289494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:56.289882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:56.290430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:56.291766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f, user.id=d04ebf64b5724be5bb236e42c18b701c) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:56.292152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:56.293648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:56.300107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 554/1103] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:56 2022] GET /identity/v3/role_assignments?user.id=d04ebf64b5724be5bb236e42c18b701c&scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:56.313360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:56.432427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f768749-52e5-4dc2-8d7b-82822555bc02 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:56.490663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:56.491324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:56.491700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:56.492666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:56.500518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:56.502079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:13:56.515556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:13:56.516519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:13:56.517419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:13:56.521837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 550/1104] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:13:56.524880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:56.666849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:56.696697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09a12f-bb68-4a1a-9648-75452a1d3c88 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:56.763161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ecd32-2e09-467f-9d1f-767e551371c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:56.764193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ecd32-2e09-467f-9d1f-767e551371c9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:56.764657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ecd32-2e09-467f-9d1f-767e551371c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:56.765123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ecd32-2e09-467f-9d1f-767e551371c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:56.766422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ecd32-2e09-467f-9d1f-767e551371c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:56.767130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ecd32-2e09-467f-9d1f-767e551371c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:56.769810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ecd32-2e09-467f-9d1f-767e551371c9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:56.794038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 555/1105] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:13:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:13:56.800474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:56.931221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a371c72-7102-4a8d-a4d7-e9da80f8ba88 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:56.933020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:56.969904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7a762-8d77-446d-85e6-21fac8ade950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:56.970637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7a762-8d77-446d-85e6-21fac8ade950 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:56.970981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7a762-8d77-446d-85e6-21fac8ade950 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:56.971314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7a762-8d77-446d-85e6-21fac8ade950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:56.972128 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7a762-8d77-446d-85e6-21fac8ade950 None admin] RBAC: Adding query filter params (role.id=eb0f55efb2424a3199b8492010e7801c, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=613e985e94c54913a5e6b93367157fc0) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:13:56.972841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7a762-8d77-446d-85e6-21fac8ade950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:56.974768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7a762-8d77-446d-85e6-21fac8ade950 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:56.983761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 551/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:13:56 2022] GET /identity/v3/role_assignments?user.id=613e985e94c54913a5e6b93367157fc0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=eb0f55efb2424a3199b8492010e7801c => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:58.517305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372f8c-d000-4c79-a1a4-37f2602329dc None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:58.517901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372f8c-d000-4c79-a1a4-37f2602329dc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:58.517901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372f8c-d000-4c79-a1a4-37f2602329dc None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:58.519581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 556/1107] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:13:58.525706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053e83-9c18-471f-9d92-fed247b00942 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:58.526298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053e83-9c18-471f-9d92-fed247b00942 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:58.526298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053e83-9c18-471f-9d92-fed247b00942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:58.546313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4d053e83-9c18-471f-9d92-fed247b00942 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:13:58.547565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4d053e83-9c18-471f-9d92-fed247b00942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:13:58.624589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 552/1108] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:13:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:13:58.898818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372f8c-d000-4c79-a1a4-37f2602329dc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:59.009098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372f8c-d000-4c79-a1a4-37f2602329dc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:59.044924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f8175-4e1e-4a31-8fc5-90be9c7caf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:59.045565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f8175-4e1e-4a31-8fc5-90be9c7caf8f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:59.045840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f8175-4e1e-4a31-8fc5-90be9c7caf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:59.046142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f8175-4e1e-4a31-8fc5-90be9c7caf8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:59.047297 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f8175-4e1e-4a31-8fc5-90be9c7caf8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:59.048735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f8175-4e1e-4a31-8fc5-90be9c7caf8f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:59.050389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 557/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:13:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:13:59.059115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d053e83-9c18-471f-9d92-fed247b00942 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:13:59.166025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:13:59.171853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d053e83-9c18-471f-9d92-fed247b00942 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:13:59.208125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175afdc7-7c54-4b76-917a-9d90f5d62d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:13:59.208748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175afdc7-7c54-4b76-917a-9d90f5d62d2d None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:13:59.209030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175afdc7-7c54-4b76-917a-9d90f5d62d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:13:59.209304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175afdc7-7c54-4b76-917a-9d90f5d62d2d None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:13:59.209977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175afdc7-7c54-4b76-917a-9d90f5d62d2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:13:59.212323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175afdc7-7c54-4b76-917a-9d90f5d62d2d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:13:59.317225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 553/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:13:59 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:14:01.340508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8dac4-8d6b-4a09-9f3b-7c5265b63fbb None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:01.341040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8dac4-8d6b-4a09-9f3b-7c5265b63fbb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:01.341040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8dac4-8d6b-4a09-9f3b-7c5265b63fbb None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:01.342436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 558/1111] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:01.347815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf006-b30d-4daf-9680-6d389b06ddc7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:01.348256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf006-b30d-4daf-9680-6d389b06ddc7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:01.348344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaf006-b30d-4daf-9680-6d389b06ddc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:01.367558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5baaf006-b30d-4daf-9680-6d389b06ddc7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:01.368761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5baaf006-b30d-4daf-9680-6d389b06ddc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:01.447942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 554/1112] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:01.770161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8dac4-8d6b-4a09-9f3b-7c5265b63fbb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:01.786888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:01.885878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8dac4-8d6b-4a09-9f3b-7c5265b63fbb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:01.924081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:01.924749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:01.925006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:01.925270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] PATH_INFO: `/v3/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:01.926738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] RBAC: Authorizing `identity:get_role(role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:01.928124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:01.929541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 559/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:01 2022] GET /identity/v3/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:01.935876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaf006-b30d-4daf-9680-6d389b06ddc7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:02.025184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:02.044617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaf006-b30d-4daf-9680-6d389b06ddc7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:02.084186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:02.084836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:02.085306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:02.085691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] PATH_INFO: `/v3/users/8e1fb1e853f4456f9464aaf51818db3d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:02.100964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] RBAC: Authorizing `identity:get_user(user_id=8e1fb1e853f4456f9464aaf51818db3d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:02.102449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:02.104214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 555/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:01 2022] GET /identity/v3/users/8e1fb1e853f4456f9464aaf51818db3d => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:02.110113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:02.196305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:02.215452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965cc7-4928-4216-bf3f-0fa79b48a0d8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:02.251539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:02.252146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:02.252419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:02.252683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:02.341520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:02.344676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:02.464322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:02.486072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 560/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:14:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:14:02.493231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:02.607106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a8dd0-2993-49d4-9437-3e8866df3c62 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:02.647358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e167aa-37f8-44a4-bda3-0e6d5e83e79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:02.648582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e167aa-37f8-44a4-bda3-0e6d5e83e79d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:02.648582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e167aa-37f8-44a4-bda3-0e6d5e83e79d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:02.649236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e167aa-37f8-44a4-bda3-0e6d5e83e79d None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:02.650346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e167aa-37f8-44a4-bda3-0e6d5e83e79d None admin] RBAC: Authorizing `identity:get_project(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:02.652060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e167aa-37f8-44a4-bda3-0e6d5e83e79d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:02.653802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 556/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:02 2022] GET /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:14:02.660822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:02.809121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c07c0-6d55-4ffa-88eb-aec539ae018d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:02.849927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9f65c-e22e-4adb-b608-0ad5e6fbc543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:02.850612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f65c-e22e-4adb-b608-0ad5e6fbc543 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:02.851649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f65c-e22e-4adb-b608-0ad5e6fbc543 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:02.852441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f65c-e22e-4adb-b608-0ad5e6fbc543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:02.853185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9f65c-e22e-4adb-b608-0ad5e6fbc543 None admin] RBAC: Adding query filter params (role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f, user.id=8e1fb1e853f4456f9464aaf51818db3d) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:02.853496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9f65c-e22e-4adb-b608-0ad5e6fbc543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:02.854982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9f65c-e22e-4adb-b608-0ad5e6fbc543 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:02.861197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 561/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:14:02 2022] GET /identity/v3/role_assignments?user.id=8e1fb1e853f4456f9464aaf51818db3d&scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:04.952950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac336-32a8-4328-a2a3-6c5584d65839 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:04.953591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac336-32a8-4328-a2a3-6c5584d65839 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:04.953591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac336-32a8-4328-a2a3-6c5584d65839 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:04.955267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 557/1118] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:04.963254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a318-43c4-4ed0-a2d2-dff23c4177eb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:04.963859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a318-43c4-4ed0-a2d2-dff23c4177eb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:04.963997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a318-43c4-4ed0-a2d2-dff23c4177eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:04.986499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ea48a318-43c4-4ed0-a2d2-dff23c4177eb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:04.988005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ea48a318-43c4-4ed0-a2d2-dff23c4177eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:05.045762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:05.084030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 562/1119] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:14:05.402472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabac336-32a8-4328-a2a3-6c5584d65839 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:05.418807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:05.534145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabac336-32a8-4328-a2a3-6c5584d65839 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:05.578160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:05.578821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:05.579100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:05.579374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] PATH_INFO: `/v3/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:05.580928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] RBAC: Authorizing `identity:get_role(role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:05.582392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:05.583923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 558/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:05 2022] GET /identity/v3/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 223 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:05.594749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a318-43c4-4ed0-a2d2-dff23c4177eb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:05.771917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a318-43c4-4ed0-a2d2-dff23c4177eb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:05.815758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:05.816486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:05.816815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:05.817137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] PATH_INFO: `/v3/users/8e1fb1e853f4456f9464aaf51818db3d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:05.818623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] RBAC: Authorizing `identity:get_user(user_id=8e1fb1e853f4456f9464aaf51818db3d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:05.821312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:05.823273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 563/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:05 2022] GET /identity/v3/users/8e1fb1e853f4456f9464aaf51818db3d => generated 286 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:05.829711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:05.918995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:05.946929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a306d-0451-4c54-baea-c8bd540c7304 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:05.988635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d8c25-f479-40eb-a949-150b66e357e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:05.989343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d8c25-f479-40eb-a949-150b66e357e9 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:05.989657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d8c25-f479-40eb-a949-150b66e357e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:05.989978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d8c25-f479-40eb-a949-150b66e357e9 None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:05.991801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d8c25-f479-40eb-a949-150b66e357e9 None admin] RBAC: Authorizing `identity:get_project(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:05.993392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d8c25-f479-40eb-a949-150b66e357e9 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:05.995210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 559/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:05 2022] GET /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:14:06.001591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:06.052888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:06.131178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ab826-ca07-4e97-b9be-b4f4231a991a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:06.196555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb939f2-db13-424d-a114-c5cd10d53f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:06.197612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb939f2-db13-424d-a114-c5cd10d53f6e None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:06.198110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb939f2-db13-424d-a114-c5cd10d53f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:06.198565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb939f2-db13-424d-a114-c5cd10d53f6e None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f/users/8e1fb1e853f4456f9464aaf51818db3d/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:06.202483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb939f2-db13-424d-a114-c5cd10d53f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f, user_id=8e1fb1e853f4456f9464aaf51818db3d, role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:06.205242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb939f2-db13-424d-a114-c5cd10d53f6e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:06.312479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 564/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:14:06 2022] PUT /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f/users/8e1fb1e853f4456f9464aaf51818db3d/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:14:08.558444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e898d-e5fd-4f1b-8d31-6a1f82803702 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:08.559297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e898d-e5fd-4f1b-8d31-6a1f82803702 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:08.559297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e898d-e5fd-4f1b-8d31-6a1f82803702 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:08.560571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 560/1124] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:14:08.567037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf8d70-bba3-4a1e-a9d8-c6968f0c5cd9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:08.567656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf8d70-bba3-4a1e-a9d8-c6968f0c5cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:08.567656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf8d70-bba3-4a1e-a9d8-c6968f0c5cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:08.589625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-efcf8d70-bba3-4a1e-a9d8-c6968f0c5cd9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:08.590896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-efcf8d70-bba3-4a1e-a9d8-c6968f0c5cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:08.673638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 565/1125] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:14:08.958500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e898d-e5fd-4f1b-8d31-6a1f82803702 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:09.100957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e898d-e5fd-4f1b-8d31-6a1f82803702 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:09.147955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:09.148617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:09.148889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:09.149156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] PATH_INFO: `/v3/roles/c4a006a615e74170bd7b09e03b5873c2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:09.150915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] RBAC: Authorizing `identity:get_role(role_id=c4a006a615e74170bd7b09e03b5873c2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:09.152321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:09.153731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 561/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:08 2022] GET /identity/v3/roles/c4a006a615e74170bd7b09e03b5873c2 => generated 223 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:09.164440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf8d70-bba3-4a1e-a9d8-c6968f0c5cd9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:09.219605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:09.336090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf8d70-bba3-4a1e-a9d8-c6968f0c5cd9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:09.390733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:09.391372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:09.391759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:09.392110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] PATH_INFO: `/v3/users/8e1fb1e853f4456f9464aaf51818db3d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:09.393373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] RBAC: Authorizing `identity:get_user(user_id=8e1fb1e853f4456f9464aaf51818db3d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:09.395097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:09.399169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 566/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:09 2022] GET /identity/v3/users/8e1fb1e853f4456f9464aaf51818db3d => generated 286 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:09.403723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:09.421753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:09.521608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4fc6e-12f4-47fa-a4a5-d9f931ed24e0 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:09.562142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:09.563191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:09.563191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:09.563556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:09.644270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:09.645886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:09.662466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:09.769652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 562/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:14:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:14:09.780410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:09.895156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:09.898564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc919c53-089f-4e6f-98eb-7d38cce846e7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:09.940376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ee407-bf23-4e80-8fae-4a57c7a8e677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:09.941173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ee407-bf23-4e80-8fae-4a57c7a8e677 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:09.941543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ee407-bf23-4e80-8fae-4a57c7a8e677 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:09.941908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ee407-bf23-4e80-8fae-4a57c7a8e677 None admin] PATH_INFO: `/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:09.943265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ee407-bf23-4e80-8fae-4a57c7a8e677 None admin] RBAC: Authorizing `identity:get_project(project_id=5b9b8b0048774a869d5e7cbdd1f02e5f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:09.944768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ee407-bf23-4e80-8fae-4a57c7a8e677 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:09.946550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 567/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:09 2022] GET /identity/v3/projects/5b9b8b0048774a869d5e7cbdd1f02e5f => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:09.953528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:10.079656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab17105-f30a-4399-a21f-def303497914 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:10.095319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:10.121658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b1c55-396e-446a-8671-c7df0eba059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:10.122322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b1c55-396e-446a-8671-c7df0eba059e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:10.122598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b1c55-396e-446a-8671-c7df0eba059e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:10.122864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b1c55-396e-446a-8671-c7df0eba059e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:10.123635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b1c55-396e-446a-8671-c7df0eba059e None admin] RBAC: Adding query filter params (role.id=c4a006a615e74170bd7b09e03b5873c2, scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f, user.id=8e1fb1e853f4456f9464aaf51818db3d) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:10.123941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b1c55-396e-446a-8671-c7df0eba059e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:10.125467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b1c55-396e-446a-8671-c7df0eba059e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:10.133038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 563/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:14:09 2022] GET /identity/v3/role_assignments?user.id=8e1fb1e853f4456f9464aaf51818db3d&scope.project.id=5b9b8b0048774a869d5e7cbdd1f02e5f&role.id=c4a006a615e74170bd7b09e03b5873c2 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:14:12.346236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451382d-b315-43b5-b0f5-bac9b4893d46 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:12.346869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451382d-b315-43b5-b0f5-bac9b4893d46 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:12.346869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451382d-b315-43b5-b0f5-bac9b4893d46 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:12.348191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 568/1131] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:14:12.354371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee290a9-45c1-44e4-a134-1e16d3d360f2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:12.355104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee290a9-45c1-44e4-a134-1e16d3d360f2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:12.355104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee290a9-45c1-44e4-a134-1e16d3d360f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:12.382547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5ee290a9-45c1-44e4-a134-1e16d3d360f2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:12.384149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5ee290a9-45c1-44e4-a134-1e16d3d360f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:12.470592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 564/1132] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:14:12.757768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451382d-b315-43b5-b0f5-bac9b4893d46 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:12.880084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451382d-b315-43b5-b0f5-bac9b4893d46 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:12.951044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c29e03-fda3-49f9-b26e-1d25c0650118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:12.952120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c29e03-fda3-49f9-b26e-1d25c0650118 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:12.952604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c29e03-fda3-49f9-b26e-1d25c0650118 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:12.953092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c29e03-fda3-49f9-b26e-1d25c0650118 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:12.955023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c29e03-fda3-49f9-b26e-1d25c0650118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:12.957682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c29e03-fda3-49f9-b26e-1d25c0650118 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:12.962842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 569/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:12.969256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee290a9-45c1-44e4-a134-1e16d3d360f2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:13.136920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee290a9-45c1-44e4-a134-1e16d3d360f2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:13.153421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:13.191780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae958de-e9a6-4fac-9ff0-eb5ed2b4f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:13.192431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae958de-e9a6-4fac-9ff0-eb5ed2b4f6ed None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:13.192714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae958de-e9a6-4fac-9ff0-eb5ed2b4f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:13.193087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae958de-e9a6-4fac-9ff0-eb5ed2b4f6ed None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:13.193660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae958de-e9a6-4fac-9ff0-eb5ed2b4f6ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:13.195219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae958de-e9a6-4fac-9ff0-eb5ed2b4f6ed None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:13.295954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 565/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:14:12 2022] POST /identity/v3/projects => generated 311 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:14:15.651560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533b1d-6e47-4ff4-bda3-6d4591ef986b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:15.652196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533b1d-6e47-4ff4-bda3-6d4591ef986b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:15.652316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533b1d-6e47-4ff4-bda3-6d4591ef986b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:15.653746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 570/1135] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:15.664737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912d027-48db-488d-b5fa-f6ee9b19d3e9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:15.665349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912d027-48db-488d-b5fa-f6ee9b19d3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:15.665743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912d027-48db-488d-b5fa-f6ee9b19d3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:15.695472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4912d027-48db-488d-b5fa-f6ee9b19d3e9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:15.697635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4912d027-48db-488d-b5fa-f6ee9b19d3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:15.782373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 566/1136] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:16.059850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533b1d-6e47-4ff4-bda3-6d4591ef986b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:16.121372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:16.197477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533b1d-6e47-4ff4-bda3-6d4591ef986b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:16.238447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19707c0d-cfc2-455c-95ad-98f2bed3c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:16.239176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19707c0d-cfc2-455c-95ad-98f2bed3c375 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:16.239510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19707c0d-cfc2-455c-95ad-98f2bed3c375 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:16.239823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19707c0d-cfc2-455c-95ad-98f2bed3c375 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:16.241251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19707c0d-cfc2-455c-95ad-98f2bed3c375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:16.242718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19707c0d-cfc2-455c-95ad-98f2bed3c375 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:16.244329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 571/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:16.251480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912d027-48db-488d-b5fa-f6ee9b19d3e9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:16.350770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:16.364256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912d027-48db-488d-b5fa-f6ee9b19d3e9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:16.401105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58f5e5-4afc-440f-84bd-a52fd73fee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:16.413818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f5e5-4afc-440f-84bd-a52fd73fee2a None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:16.413818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f5e5-4afc-440f-84bd-a52fd73fee2a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:16.413818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f5e5-4afc-440f-84bd-a52fd73fee2a None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:16.413818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58f5e5-4afc-440f-84bd-a52fd73fee2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:16.413818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58f5e5-4afc-440f-84bd-a52fd73fee2a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:16.694805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 567/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:14:16 2022] POST /identity/v3/users => generated 286 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:14:18.963841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31865046-c26e-4519-9c27-6c9636240dce None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:18.964478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31865046-c26e-4519-9c27-6c9636240dce None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:18.964478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31865046-c26e-4519-9c27-6c9636240dce None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:18.965855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 572/1139] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:14:18.972132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51926d40-3fc3-4056-99ad-64cf979a2a79 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:18.972709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51926d40-3fc3-4056-99ad-64cf979a2a79 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:18.972709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51926d40-3fc3-4056-99ad-64cf979a2a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:18.994225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-51926d40-3fc3-4056-99ad-64cf979a2a79 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:18.995451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-51926d40-3fc3-4056-99ad-64cf979a2a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:19.085420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 568/1140] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:19.404479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31865046-c26e-4519-9c27-6c9636240dce None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:19.539266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31865046-c26e-4519-9c27-6c9636240dce None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:19.542368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:19.578578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:19.579236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:19.579513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:19.579784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:19.592569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:14:19.597717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:14:19.602237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:14:19.603880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 573/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:14:19.606537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51926d40-3fc3-4056-99ad-64cf979a2a79 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:19.692406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:19.719878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51926d40-3fc3-4056-99ad-64cf979a2a79 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:19.759496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:19.760132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:19.760410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:19.760681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:19.761372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:19.761673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:19.763131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:19.773329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 569/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:19.780542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:19.924012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f640e1-a55e-4297-99c2-fe5d851bbbc5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:19.964694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:19.965394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:19.965662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:19.965838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] PATH_INFO: `/v3/users/e02363ad8f334eb58f0392a10a59b159` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:19.982269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=e02363ad8f334eb58f0392a10a59b159)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:19.983826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:19.985638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 574/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:19 2022] GET /identity/v3/users/e02363ad8f334eb58f0392a10a59b159 => generated 286 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:19.992331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:20.116655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941d838-f660-4e21-b446-5e429bbed9a8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:20.159822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:20.160453 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:20.160727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:20.160996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:20.165341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:20.246121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:20.247545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:20.421342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 570/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:14:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:14:20.427702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:20.457195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:20.554375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192b589-c678-4d45-857c-59b23eaa8fe4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:20.595549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ed952-3f3b-4156-80eb-a59775b62fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:20.596383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed952-3f3b-4156-80eb-a59775b62fff None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:20.596665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed952-3f3b-4156-80eb-a59775b62fff None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:20.596954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed952-3f3b-4156-80eb-a59775b62fff None admin] PATH_INFO: `/v3/projects/3aad8e5bc466441e86102dcb55028ef3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:20.598220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ed952-3f3b-4156-80eb-a59775b62fff None admin] RBAC: Authorizing `identity:get_project(project_id=3aad8e5bc466441e86102dcb55028ef3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:20.600269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ed952-3f3b-4156-80eb-a59775b62fff None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:20.602304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 575/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:20 2022] GET /identity/v3/projects/3aad8e5bc466441e86102dcb55028ef3 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:20.608995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:20.652053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:20.724551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9914fd0-8d2b-4814-8055-c9b1ca3ba588 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:20.763509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2570e4-ade3-4a25-871f-ae73347ef57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:20.764110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2570e4-ade3-4a25-871f-ae73347ef57e None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:20.764387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2570e4-ade3-4a25-871f-ae73347ef57e None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:20.764658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2570e4-ade3-4a25-871f-ae73347ef57e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:20.765385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2570e4-ade3-4a25-871f-ae73347ef57e None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=3aad8e5bc466441e86102dcb55028ef3, user.id=e02363ad8f334eb58f0392a10a59b159) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:20.765692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2570e4-ade3-4a25-871f-ae73347ef57e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:20.767150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2570e4-ade3-4a25-871f-ae73347ef57e None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:20.773618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 571/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:14:20 2022] GET /identity/v3/role_assignments?user.id=e02363ad8f334eb58f0392a10a59b159&scope.project.id=3aad8e5bc466441e86102dcb55028ef3&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:14:22.855738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1f29e-be11-4fbf-9600-af699cb68de4 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:22.856317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1f29e-be11-4fbf-9600-af699cb68de4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:22.856317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1f29e-be11-4fbf-9600-af699cb68de4 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:22.857675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 576/1147] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:22.863521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3961e75-99d9-49f5-a10d-d9ac515a4955 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:22.864027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3961e75-99d9-49f5-a10d-d9ac515a4955 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:22.864027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3961e75-99d9-49f5-a10d-d9ac515a4955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:22.884229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e3961e75-99d9-49f5-a10d-d9ac515a4955 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:22.885439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e3961e75-99d9-49f5-a10d-d9ac515a4955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:22.961575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 572/1148] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:23.236359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1f29e-be11-4fbf-9600-af699cb68de4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:23.331618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:23.342624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1f29e-be11-4fbf-9600-af699cb68de4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:23.379689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:23.380326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:23.380604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:23.380921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:23.391528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:14:23.396254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:14:23.397962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:14:23.398985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 577/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:14:23.404984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3961e75-99d9-49f5-a10d-d9ac515a4955 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:23.435501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:23.507609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3961e75-99d9-49f5-a10d-d9ac515a4955 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:23.551389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:23.552013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:23.552291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:23.552563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:23.553462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:23.553765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:23.555187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:23.563986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 573/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:14:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:23.570792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:23.675572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742a97-59be-47e0-b0bf-0fe6ea110058 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:23.714667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:23.715302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:23.715574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:23.715840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] PATH_INFO: `/v3/users/e02363ad8f334eb58f0392a10a59b159` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:23.716973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] RBAC: Authorizing `identity:get_user(user_id=e02363ad8f334eb58f0392a10a59b159)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:23.718426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:23.720036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 578/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:23 2022] GET /identity/v3/users/e02363ad8f334eb58f0392a10a59b159 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:23.726175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:23.824625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:23.833448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ad482-c744-44b7-8d04-37249c9e6353 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:23.870165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329297a6-8f68-4609-9879-f4172864ba2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:23.870902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329297a6-8f68-4609-9879-f4172864ba2a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:23.871049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329297a6-8f68-4609-9879-f4172864ba2a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:23.871310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329297a6-8f68-4609-9879-f4172864ba2a None admin] PATH_INFO: `/v3/projects/3aad8e5bc466441e86102dcb55028ef3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:23.872473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329297a6-8f68-4609-9879-f4172864ba2a None admin] RBAC: Authorizing `identity:get_project(project_id=3aad8e5bc466441e86102dcb55028ef3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:23.873916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329297a6-8f68-4609-9879-f4172864ba2a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:23.875639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 574/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:23 2022] GET /identity/v3/projects/3aad8e5bc466441e86102dcb55028ef3 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:23.882137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:23.905666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:23.993164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff11c0e-6361-4acc-979c-924248f5bdbe None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:24.029290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2d1f3-8a1b-4bc9-b335-cbeca219c8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:24.029962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2d1f3-8a1b-4bc9-b335-cbeca219c8df None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:24.030272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2d1f3-8a1b-4bc9-b335-cbeca219c8df None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:24.030544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2d1f3-8a1b-4bc9-b335-cbeca219c8df None admin] PATH_INFO: `/v3/projects/3aad8e5bc466441e86102dcb55028ef3/users/e02363ad8f334eb58f0392a10a59b159/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:24.032467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2d1f3-8a1b-4bc9-b335-cbeca219c8df None admin] RBAC: Authorizing `identity:create_grant(project_id=3aad8e5bc466441e86102dcb55028ef3, user_id=e02363ad8f334eb58f0392a10a59b159, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:24.033892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2d1f3-8a1b-4bc9-b335-cbeca219c8df None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:24.121632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 579/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:14:23 2022] PUT /identity/v3/projects/3aad8e5bc466441e86102dcb55028ef3/users/e02363ad8f334eb58f0392a10a59b159/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:14:26.069796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214dd8-392d-43d3-8033-094adf90234c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:26.070407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214dd8-392d-43d3-8033-094adf90234c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:26.070407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214dd8-392d-43d3-8033-094adf90234c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:26.071725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 575/1154] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:26.077841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c377225-0207-4ab3-a4d3-421b794ed6d3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:26.078364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c377225-0207-4ab3-a4d3-421b794ed6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:26.078523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c377225-0207-4ab3-a4d3-421b794ed6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:26.098645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5c377225-0207-4ab3-a4d3-421b794ed6d3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:26.100448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5c377225-0207-4ab3-a4d3-421b794ed6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:26.174054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 580/1155] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:26.453480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214dd8-392d-43d3-8033-094adf90234c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:26.556458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214dd8-392d-43d3-8033-094adf90234c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:26.589534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:26.590179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:26.590457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:26.590729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:26.601099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:14:26.606634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:14:26.607769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:14:26.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 576/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:14:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:14:26.614638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c377225-0207-4ab3-a4d3-421b794ed6d3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:26.626548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:26.724215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c377225-0207-4ab3-a4d3-421b794ed6d3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:26.761915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:26.762565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:26.762845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:26.763119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:26.763960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:26.764284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:26.765686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:26.775568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 581/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:14:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:26.781688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:26.805964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:26.886689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c1126-ca98-4358-97f9-433b5a041e87 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:26.922949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:26.923582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:26.923861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:26.924139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] PATH_INFO: `/v3/users/e02363ad8f334eb58f0392a10a59b159` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:26.925261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] RBAC: Authorizing `identity:get_user(user_id=e02363ad8f334eb58f0392a10a59b159)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:26.926728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:26.928706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 577/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:26 2022] GET /identity/v3/users/e02363ad8f334eb58f0392a10a59b159 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:26.934774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:27.017010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:27.049950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfbd11-38ee-4eb1-b0fe-e0171bcee3ae None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:27.083708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:27.085131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:27.085131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:27.085368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:27.162939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:27.164432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:27.234544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:27.266331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 582/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:14:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:14:27.274745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:27.367653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:27.381261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f17358-9d0c-41d0-9729-697b5c74a8c2 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:27.419211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0247fad-74df-4135-a7b0-24e73527f96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:27.419759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247fad-74df-4135-a7b0-24e73527f96a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:27.420034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247fad-74df-4135-a7b0-24e73527f96a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:27.420300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247fad-74df-4135-a7b0-24e73527f96a None admin] PATH_INFO: `/v3/projects/3aad8e5bc466441e86102dcb55028ef3` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:27.421447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0247fad-74df-4135-a7b0-24e73527f96a None admin] RBAC: Authorizing `identity:get_project(project_id=3aad8e5bc466441e86102dcb55028ef3)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:27.422918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0247fad-74df-4135-a7b0-24e73527f96a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:27.424622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 578/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:27 2022] GET /identity/v3/projects/3aad8e5bc466441e86102dcb55028ef3 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:27.431256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:27.542372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea77ecc-a6c6-4ba7-9155-b1b33d66d6e1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:27.580927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458328a0-3f77-465c-8660-bc4502aacd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:27.581540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458328a0-3f77-465c-8660-bc4502aacd19 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:27.581817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458328a0-3f77-465c-8660-bc4502aacd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:27.582125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458328a0-3f77-465c-8660-bc4502aacd19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:27.582866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458328a0-3f77-465c-8660-bc4502aacd19 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=3aad8e5bc466441e86102dcb55028ef3, user.id=e02363ad8f334eb58f0392a10a59b159) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:27.583172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458328a0-3f77-465c-8660-bc4502aacd19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:27.585413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458328a0-3f77-465c-8660-bc4502aacd19 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:27.591968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 583/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:14:27 2022] GET /identity/v3/role_assignments?user.id=e02363ad8f334eb58f0392a10a59b159&scope.project.id=3aad8e5bc466441e86102dcb55028ef3&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:14:29.566361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef472661-944b-48a2-a401-b24b99fb362a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:29.566928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef472661-944b-48a2-a401-b24b99fb362a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:29.566928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef472661-944b-48a2-a401-b24b99fb362a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:29.568791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 579/1162] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:29.577254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6631a9-1458-4d96-8d66-cfa84db75102 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:29.577825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6631a9-1458-4d96-8d66-cfa84db75102 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:29.577825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6631a9-1458-4d96-8d66-cfa84db75102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:29.596814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0f6631a9-1458-4d96-8d66-cfa84db75102 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:29.598055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0f6631a9-1458-4d96-8d66-cfa84db75102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:29.627159 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:29.673961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 584/1163] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:14:29.949570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef472661-944b-48a2-a401-b24b99fb362a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:30.062398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef472661-944b-48a2-a401-b24b99fb362a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:30.099640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:30.100237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:30.100500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:30.100800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:30.127438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:30.173764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:30.175273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:30.291418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 580/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:14:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:14:30.300941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6631a9-1458-4d96-8d66-cfa84db75102 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:30.420773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6631a9-1458-4d96-8d66-cfa84db75102 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:30.459808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d70bb9-f616-45e2-8995-9de3f8c05e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:30.460583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d70bb9-f616-45e2-8995-9de3f8c05e37 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:30.460972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d70bb9-f616-45e2-8995-9de3f8c05e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:30.461335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d70bb9-f616-45e2-8995-9de3f8c05e37 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:30.469267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d70bb9-f616-45e2-8995-9de3f8c05e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:30.470763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d70bb9-f616-45e2-8995-9de3f8c05e37 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-75d70bb9-f616-45e2-8995-9de3f8c05e37 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:14:30.483285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:14:30.484806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:14:30.485924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 585/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:14:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:14:30.491335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:30.577240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:30.623443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9d46b-753f-4b8f-9bb5-e09dba6fe60d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:30.658360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96355ca2-2714-4ab4-81e3-2a2dc54e6733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:30.658972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355ca2-2714-4ab4-81e3-2a2dc54e6733 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:30.659250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355ca2-2714-4ab4-81e3-2a2dc54e6733 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:30.659516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96355ca2-2714-4ab4-81e3-2a2dc54e6733 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:30.660167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96355ca2-2714-4ab4-81e3-2a2dc54e6733 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:30.660468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96355ca2-2714-4ab4-81e3-2a2dc54e6733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:30.662513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96355ca2-2714-4ab4-81e3-2a2dc54e6733 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:30.670431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 581/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 24 14:14:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:32.667174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90dbaa-d605-4b31-82f1-aec8f8d03a52 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:32.667702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90dbaa-d605-4b31-82f1-aec8f8d03a52 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:32.668029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90dbaa-d605-4b31-82f1-aec8f8d03a52 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:32.670567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 586/1167] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:32.676761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00d44f-df80-4b57-aaf6-c1fb651f6cd6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:32.677358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00d44f-df80-4b57-aaf6-c1fb651f6cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:32.677358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00d44f-df80-4b57-aaf6-c1fb651f6cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:32.698535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-dc00d44f-df80-4b57-aaf6-c1fb651f6cd6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:32.699778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dc00d44f-df80-4b57-aaf6-c1fb651f6cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:32.779470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 582/1168] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:33.056305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90dbaa-d605-4b31-82f1-aec8f8d03a52 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:33.084336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:33.169417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90dbaa-d605-4b31-82f1-aec8f8d03a52 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:33.209275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6edd33-f8c2-49bf-b3b2-218e4074c88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:33.209897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edd33-f8c2-49bf-b3b2-218e4074c88b None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:33.210202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edd33-f8c2-49bf-b3b2-218e4074c88b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:33.210526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edd33-f8c2-49bf-b3b2-218e4074c88b None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:33.211202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6edd33-f8c2-49bf-b3b2-218e4074c88b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:33.212579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6edd33-f8c2-49bf-b3b2-218e4074c88b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:33.345647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 587/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Wed Aug 24 14:14:33 2022] POST /identity/v3/domains => generated 267 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:14:35.372251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b74a7-20c7-4bca-b798-5dab8dabdf12 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:35.373065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b74a7-20c7-4bca-b798-5dab8dabdf12 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:35.373270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b74a7-20c7-4bca-b798-5dab8dabdf12 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:35.374975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 583/1170] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:35.382805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc676000-fdf8-49ed-8136-3a6612a64b94 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:35.383354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc676000-fdf8-49ed-8136-3a6612a64b94 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:35.383767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc676000-fdf8-49ed-8136-3a6612a64b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:35.404824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dc676000-fdf8-49ed-8136-3a6612a64b94 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:35.406246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dc676000-fdf8-49ed-8136-3a6612a64b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:35.484676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 588/1171] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:35.760096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b74a7-20c7-4bca-b798-5dab8dabdf12 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:35.822801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:35.883375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b74a7-20c7-4bca-b798-5dab8dabdf12 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:35.923495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5f359-5fa4-4b35-99e3-fd73c2c77804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:35.924122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5f359-5fa4-4b35-99e3-fd73c2c77804 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:35.924398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5f359-5fa4-4b35-99e3-fd73c2c77804 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:35.924697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5f359-5fa4-4b35-99e3-fd73c2c77804 None admin] PATH_INFO: `/v3/domains/6f5d664e08ee4df18e9e9478d6d31d12` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:35.936705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5f359-5fa4-4b35-99e3-fd73c2c77804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6f5d664e08ee4df18e9e9478d6d31d12)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:35.938205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5f359-5fa4-4b35-99e3-fd73c2c77804 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:35.939912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 584/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Aug 24 14:14:35 2022] GET /identity/v3/domains/6f5d664e08ee4df18e9e9478d6d31d12 => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:14:35.946819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc676000-fdf8-49ed-8136-3a6612a64b94 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:35.963775 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:36.064157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc676000-fdf8-49ed-8136-3a6612a64b94 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:36.104947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba4888-fa4a-42e4-b76e-dd977e423958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:36.105569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba4888-fa4a-42e4-b76e-dd977e423958 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:36.105845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba4888-fa4a-42e4-b76e-dd977e423958 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:36.106147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba4888-fa4a-42e4-b76e-dd977e423958 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:36.106810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba4888-fa4a-42e4-b76e-dd977e423958 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:36.108490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba4888-fa4a-42e4-b76e-dd977e423958 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:36.219688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 589/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:14:35 2022] POST /identity/v3/projects => generated 361 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:14:38.397754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0eefa-6775-4886-9834-b6b8233442e3 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:38.398469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0eefa-6775-4886-9834-b6b8233442e3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:38.398469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0eefa-6775-4886-9834-b6b8233442e3 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:38.399693 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 585/1174] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:38.405800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2dc26-3c28-47ac-8504-99e7fc0c6fd1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:38.406436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2dc26-3c28-47ac-8504-99e7fc0c6fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:38.406436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2dc26-3c28-47ac-8504-99e7fc0c6fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:38.425656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e3b2dc26-3c28-47ac-8504-99e7fc0c6fd1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:38.427076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e3b2dc26-3c28-47ac-8504-99e7fc0c6fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:38.493825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:38.501875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 590/1175] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:38.773213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0eefa-6775-4886-9834-b6b8233442e3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:38.904089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0eefa-6775-4886-9834-b6b8233442e3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:38.916948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:38.943856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb86322-f18b-4c73-98a8-73a0dc7be025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:38.944706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb86322-f18b-4c73-98a8-73a0dc7be025 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:38.944986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb86322-f18b-4c73-98a8-73a0dc7be025 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:38.945257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb86322-f18b-4c73-98a8-73a0dc7be025 None admin] PATH_INFO: `/v3/domains/6f5d664e08ee4df18e9e9478d6d31d12` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:38.946507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb86322-f18b-4c73-98a8-73a0dc7be025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6f5d664e08ee4df18e9e9478d6d31d12)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:38.948003 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb86322-f18b-4c73-98a8-73a0dc7be025 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:38.950092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 586/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Aug 24 14:14:38 2022] GET /identity/v3/domains/6f5d664e08ee4df18e9e9478d6d31d12 => generated 267 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:38.956233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2dc26-3c28-47ac-8504-99e7fc0c6fd1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:39.076416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2dc26-3c28-47ac-8504-99e7fc0c6fd1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:39.116984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8118333-9f2a-4db4-a2fb-a07270e27963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:39.117644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8118333-9f2a-4db4-a2fb-a07270e27963 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:39.117906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8118333-9f2a-4db4-a2fb-a07270e27963 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:39.118212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8118333-9f2a-4db4-a2fb-a07270e27963 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:39.118892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8118333-9f2a-4db4-a2fb-a07270e27963 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:39.120374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8118333-9f2a-4db4-a2fb-a07270e27963 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:39.235539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 591/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:14:38 2022] POST /identity/v3/users => generated 311 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:14:41.394746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ab3d5-6368-48e8-a557-6f2289e9252c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:41.395380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ab3d5-6368-48e8-a557-6f2289e9252c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:41.395380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ab3d5-6368-48e8-a557-6f2289e9252c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:41.397488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 587/1178] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:14:41.403957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496576ca-5eaa-4ec9-a3e0-c59e30745366 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:41.404573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496576ca-5eaa-4ec9-a3e0-c59e30745366 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:41.404573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496576ca-5eaa-4ec9-a3e0-c59e30745366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:41.424818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-496576ca-5eaa-4ec9-a3e0-c59e30745366 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:41.426099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-496576ca-5eaa-4ec9-a3e0-c59e30745366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:41.488651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:41.512295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 592/1179] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:41.819968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ab3d5-6368-48e8-a557-6f2289e9252c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:41.938369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ab3d5-6368-48e8-a557-6f2289e9252c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:41.977655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:41.978333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:41.978614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:41.978887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:41.991286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:14:41.996460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:14:41.998356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:14:41.999888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 588/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:14:42.005593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496576ca-5eaa-4ec9-a3e0-c59e30745366 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:42.124760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496576ca-5eaa-4ec9-a3e0-c59e30745366 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:42.166052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:42.166692 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:42.166970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:42.167243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:42.167936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:42.168242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:42.169638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:42.179408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 593/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:42.189020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:42.260481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:42.352977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cb45d-2ce5-427b-9865-6cb0b5672bbc None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:42.395122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:42.396469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:42.397234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:42.397520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] PATH_INFO: `/v3/users/089950e8aa634e7c81f9b9b1e7581a47` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:42.414509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] RBAC: Authorizing `identity:get_user(user_id=089950e8aa634e7c81f9b9b1e7581a47)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:42.416009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:42.417799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 589/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:42 2022] GET /identity/v3/users/089950e8aa634e7c81f9b9b1e7581a47 => generated 311 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:42.424278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:42.479042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:42.544127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23856148-e31e-440b-8bc2-6ccc3107f73e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:42.584525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:42.585157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:42.585439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:42.585713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:42.664316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:42.665709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:42.719286 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:42.785644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 594/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:14:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:14:42.793431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:42.928677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:42.937445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b8b2e-dd86-4546-b477-537a60f7b83c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:42.982631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb96ac-86d3-4a11-81ba-b47d690050bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:42.983345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb96ac-86d3-4a11-81ba-b47d690050bc None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:42.983672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb96ac-86d3-4a11-81ba-b47d690050bc None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:42.983993 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb96ac-86d3-4a11-81ba-b47d690050bc None admin] PATH_INFO: `/v3/projects/1785e78cbc744bb5ac78855fec734f2e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:42.985276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb96ac-86d3-4a11-81ba-b47d690050bc None admin] RBAC: Authorizing `identity:get_project(project_id=1785e78cbc744bb5ac78855fec734f2e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:42.987044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb96ac-86d3-4a11-81ba-b47d690050bc None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:42.988817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 590/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:42 2022] GET /identity/v3/projects/1785e78cbc744bb5ac78855fec734f2e => generated 361 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:42.996047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:43.175965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfa1ed-89ee-4465-93a2-4d561d731660 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:43.237419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764888ea-0be2-4b5a-894e-adc7dfd3af27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:43.238408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764888ea-0be2-4b5a-894e-adc7dfd3af27 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:43.238875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764888ea-0be2-4b5a-894e-adc7dfd3af27 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:43.239345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764888ea-0be2-4b5a-894e-adc7dfd3af27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:43.240221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764888ea-0be2-4b5a-894e-adc7dfd3af27 None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=1785e78cbc744bb5ac78855fec734f2e, user.id=089950e8aa634e7c81f9b9b1e7581a47) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:43.240641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764888ea-0be2-4b5a-894e-adc7dfd3af27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:43.242760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764888ea-0be2-4b5a-894e-adc7dfd3af27 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:43.250291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 595/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:14:42 2022] GET /identity/v3/role_assignments?user.id=089950e8aa634e7c81f9b9b1e7581a47&scope.project.id=1785e78cbc744bb5ac78855fec734f2e&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:45.220295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14155ac-8a56-4af3-b4d3-d2e5a7c5249b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:45.220992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14155ac-8a56-4af3-b4d3-d2e5a7c5249b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:45.220992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14155ac-8a56-4af3-b4d3-d2e5a7c5249b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:45.222554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 591/1186] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:14:45.231900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe35cf3-b1b1-46f5-97f2-f04c3b44f0b0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:45.232505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe35cf3-b1b1-46f5-97f2-f04c3b44f0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:45.232505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe35cf3-b1b1-46f5-97f2-f04c3b44f0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:45.255453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7fe35cf3-b1b1-46f5-97f2-f04c3b44f0b0 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:45.257232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7fe35cf3-b1b1-46f5-97f2-f04c3b44f0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:45.260087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:45.346748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 596/1187] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:14:45.637174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14155ac-8a56-4af3-b4d3-d2e5a7c5249b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:45.760687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14155ac-8a56-4af3-b4d3-d2e5a7c5249b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:45.800520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:45.801174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:45.801453 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:45.802385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:45.814977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:14:45.820597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:14:45.822373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:14:45.824231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 592/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:14:45.831895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe35cf3-b1b1-46f5-97f2-f04c3b44f0b0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:46.027847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe35cf3-b1b1-46f5-97f2-f04c3b44f0b0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:46.092583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:46.093219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:46.093499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:46.093771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:46.094563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:46.094871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:46.096265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:46.103380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:46.107243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 597/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:46.113474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:46.143819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:46.230835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3352091-ccf1-462b-aea1-d4505a185864 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:46.270009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:46.271450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:46.271939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:46.272319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] PATH_INFO: `/v3/users/089950e8aa634e7c81f9b9b1e7581a47` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:46.273575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] RBAC: Authorizing `identity:get_user(user_id=089950e8aa634e7c81f9b9b1e7581a47)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:46.275122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:46.276852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 593/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:46 2022] GET /identity/v3/users/089950e8aa634e7c81f9b9b1e7581a47 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:46.283367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:46.403388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5aa137-0d6a-436e-b5b6-99d243014d6c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:46.444093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20123a8b-6b57-484d-879d-9320730105bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:46.444793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20123a8b-6b57-484d-879d-9320730105bd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:46.445113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20123a8b-6b57-484d-879d-9320730105bd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:46.445431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20123a8b-6b57-484d-879d-9320730105bd None admin] PATH_INFO: `/v3/projects/1785e78cbc744bb5ac78855fec734f2e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:46.446807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20123a8b-6b57-484d-879d-9320730105bd None admin] RBAC: Authorizing `identity:get_project(project_id=1785e78cbc744bb5ac78855fec734f2e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:46.448305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20123a8b-6b57-484d-879d-9320730105bd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:46.450087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 598/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:46 2022] GET /identity/v3/projects/1785e78cbc744bb5ac78855fec734f2e => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:46.461126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:46.583421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:46.596918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bce8e-9c07-4843-a3f3-dd31f5afad77 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:46.641393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c5bd-4feb-4e1a-8bf4-6a37bbad4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:46.642258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c5bd-4feb-4e1a-8bf4-6a37bbad4ee2 None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:46.642601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c5bd-4feb-4e1a-8bf4-6a37bbad4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:46.642937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c5bd-4feb-4e1a-8bf4-6a37bbad4ee2 None admin] PATH_INFO: `/v3/projects/1785e78cbc744bb5ac78855fec734f2e/users/089950e8aa634e7c81f9b9b1e7581a47/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:46.644954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c5bd-4feb-4e1a-8bf4-6a37bbad4ee2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1785e78cbc744bb5ac78855fec734f2e, user_id=089950e8aa634e7c81f9b9b1e7581a47, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:46.646552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c5bd-4feb-4e1a-8bf4-6a37bbad4ee2 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:46.755943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 594/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:14:46 2022] PUT /identity/v3/projects/1785e78cbc744bb5ac78855fec734f2e/users/089950e8aa634e7c81f9b9b1e7581a47/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:14:48.876230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07487f33-c166-4bdb-9e99-083d009d6cef None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:48.876842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07487f33-c166-4bdb-9e99-083d009d6cef None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:48.876842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07487f33-c166-4bdb-9e99-083d009d6cef None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:48.878155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 599/1193] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:14:48.884912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88958ce0-9202-4cac-849c-49c14344caa2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:48.885508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88958ce0-9202-4cac-849c-49c14344caa2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:48.885632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88958ce0-9202-4cac-849c-49c14344caa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:48.905354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-88958ce0-9202-4cac-849c-49c14344caa2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:14:48.906756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-88958ce0-9202-4cac-849c-49c14344caa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:14:48.986230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 595/1194] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:14:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:14:49.265896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07487f33-c166-4bdb-9e99-083d009d6cef None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:49.345928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:49.388903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07487f33-c166-4bdb-9e99-083d009d6cef None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:49.429876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:49.430565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:49.430914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:49.431258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:49.443831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:14:49.448780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:14:49.450609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:14:49.452290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 600/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:14:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:14:49.461446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88958ce0-9202-4cac-849c-49c14344caa2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:49.570314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:49.590587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88958ce0-9202-4cac-849c-49c14344caa2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:49.638470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:49.639096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:49.639377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:49.639641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:49.640326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:49.640625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:49.642049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:49.654485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 596/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:49.661426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:49.784428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64df7a9-49f5-4a77-8c55-2732e4a2de7e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:49.818682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:49.840007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:49.840007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:49.840007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:49.840007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] PATH_INFO: `/v3/users/089950e8aa634e7c81f9b9b1e7581a47` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:49.840605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] RBAC: Authorizing `identity:get_user(user_id=089950e8aa634e7c81f9b9b1e7581a47)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:49.840605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:49.840605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 601/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 24 14:14:49 2022] GET /identity/v3/users/089950e8aa634e7c81f9b9b1e7581a47 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:49.842966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:49.962502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1bee-189d-4737-a151-0f1df79870ad None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:50.003130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:50.003835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:50.004124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:50.004390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:50.023917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:50.089478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:50.091127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:50.225746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 597/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:14:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 14:14:50.232200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:50.354149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75038a-5ec6-4cce-90a6-b5541626d653 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:50.407805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91ab50-005c-4518-aab7-292fa1817eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:50.408898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91ab50-005c-4518-aab7-292fa1817eeb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:50.409378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91ab50-005c-4518-aab7-292fa1817eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:50.409860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91ab50-005c-4518-aab7-292fa1817eeb None admin] PATH_INFO: `/v3/projects/1785e78cbc744bb5ac78855fec734f2e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:50.413284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91ab50-005c-4518-aab7-292fa1817eeb None admin] RBAC: Authorizing `identity:get_project(project_id=1785e78cbc744bb5ac78855fec734f2e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:50.416013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91ab50-005c-4518-aab7-292fa1817eeb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:50.419309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 602/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 24 14:14:50 2022] GET /identity/v3/projects/1785e78cbc744bb5ac78855fec734f2e => generated 361 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:14:50.426204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:14:50.481601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:14:50.546180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b4ef0-dcca-49e4-9b91-74e70d4cc072 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:14:50.587382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300f8c1-d8dc-45d3-9a40-13adfe5b5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:14:50.588015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f8c1-d8dc-45d3-9a40-13adfe5b5a3b None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:14:50.588293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f8c1-d8dc-45d3-9a40-13adfe5b5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:14:50.588717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f8c1-d8dc-45d3-9a40-13adfe5b5a3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:14:50.589627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300f8c1-d8dc-45d3-9a40-13adfe5b5a3b None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=1785e78cbc744bb5ac78855fec734f2e, user.id=089950e8aa634e7c81f9b9b1e7581a47) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:14:50.589968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300f8c1-d8dc-45d3-9a40-13adfe5b5a3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:14:50.591793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300f8c1-d8dc-45d3-9a40-13adfe5b5a3b None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:14:50.599830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 598/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:14:50 2022] GET /identity/v3/role_assignments?user.id=089950e8aa634e7c81f9b9b1e7581a47&scope.project.id=1785e78cbc744bb5ac78855fec734f2e&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:15:00.400179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=68 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:15:00.596228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=46 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:17.686788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43a2a-a0ac-4551-bb74-a4dae2dea9a0 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:17.687684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43a2a-a0ac-4551-bb74-a4dae2dea9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:17.687684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43a2a-a0ac-4551-bb74-a4dae2dea9a0 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:17.690076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 603/1201] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:16:17.699135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f6dce-b1f5-4e0c-ae7f-e0c535586630 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:17.699783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f6dce-b1f5-4e0c-ae7f-e0c535586630 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:17.699783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f6dce-b1f5-4e0c-ae7f-e0c535586630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:17.719397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-146f6dce-b1f5-4e0c-ae7f-e0c535586630 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:17.720719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-146f6dce-b1f5-4e0c-ae7f-e0c535586630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:17.804486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 599/1202] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:16:18.090315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43a2a-a0ac-4551-bb74-a4dae2dea9a0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:18.237153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43a2a-a0ac-4551-bb74-a4dae2dea9a0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:18.280972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:18.281924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:18.281924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:18.282281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:18.306532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:18.308163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:18.326173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:18.328073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:18.329936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:18.329936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:18.329936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:18.329936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 604/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:18.335663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f6dce-b1f5-4e0c-ae7f-e0c535586630 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:18.490124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f6dce-b1f5-4e0c-ae7f-e0c535586630 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:18.529395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:18.536488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb53ca4-75e7-4829-a7a3-186b78d6c2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:18.539811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53ca4-75e7-4829-a7a3-186b78d6c2ca None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:18.539811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53ca4-75e7-4829-a7a3-186b78d6c2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:18.539811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53ca4-75e7-4829-a7a3-186b78d6c2ca None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:18.539811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53ca4-75e7-4829-a7a3-186b78d6c2ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:18.539811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53ca4-75e7-4829-a7a3-186b78d6c2ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:18.540413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53ca4-75e7-4829-a7a3-186b78d6c2ca None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:18.554174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 600/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:18.563821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:18.627630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:18.682319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3f17c-6b37-4621-8a14-507e12344395 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:18.728232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bf5e7-4ef6-4268-bd33-6dd71e129e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:18.729308 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf5e7-4ef6-4268-bd33-6dd71e129e85 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:18.729802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf5e7-4ef6-4268-bd33-6dd71e129e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:18.730370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf5e7-4ef6-4268-bd33-6dd71e129e85 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:18.731589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bf5e7-4ef6-4268-bd33-6dd71e129e85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:18.734449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bf5e7-4ef6-4268-bd33-6dd71e129e85 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:18.875343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 605/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 24 14:16:18 2022] POST /identity/v3/users => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:16:20.940055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec5e9-e416-4a38-9b17-db0304f7a380 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:20.940708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec5e9-e416-4a38-9b17-db0304f7a380 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:20.940708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec5e9-e416-4a38-9b17-db0304f7a380 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:20.942063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 601/1206] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:16:20.951650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490544c5-da08-4e88-9223-04bff9f5977a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:20.952217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490544c5-da08-4e88-9223-04bff9f5977a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:20.952315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490544c5-da08-4e88-9223-04bff9f5977a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:20.971804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-490544c5-da08-4e88-9223-04bff9f5977a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:20.973652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-490544c5-da08-4e88-9223-04bff9f5977a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:21.073174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 606/1207] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:16:21.354579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec5e9-e416-4a38-9b17-db0304f7a380 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:21.511082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec5e9-e416-4a38-9b17-db0304f7a380 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:21.552857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:21.553512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:21.553804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:21.554087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:21.566613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:16:21.572609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:21.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:21.576000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 602/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:21.581033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490544c5-da08-4e88-9223-04bff9f5977a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:21.631134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:21.701866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490544c5-da08-4e88-9223-04bff9f5977a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:21.742216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:21.742892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:21.743215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:21.743560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:21.744334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:21.744682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:21.746186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:21.756576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 607/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:16:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:21.762987 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:21.824839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:21.881804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b6494-358f-4508-868e-fbc2eb0447e6 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:21.920194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:21.920843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:21.921118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:21.921386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:21.933931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:21.935587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:21.955294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:21.957098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:21.959855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:21.959855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:21.959855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:21.959855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 603/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:21.964451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:22.102675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:22.105955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd5dd3-e72f-4447-a58c-b0cd44e67f7b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:22.147834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:22.148346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:22.148615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:22.148883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:22.149557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:22.149859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:22.151301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:22.167068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 608/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:22.172839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:22.299488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b0523-9210-4911-bf71-774311a41c87 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:22.301388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:22.341541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:22.342395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:22.342776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:22.343181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:22.352979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:22.354711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:16:22.369674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:22.371556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:22.373324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 604/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:22.377773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:22.501864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2f5e4-99ef-4fdf-9cde-0cdcb9718477 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:22.544275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:22.544971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:22.545192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:22.545464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:22.546273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:22.546581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:22.548134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:22.571327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 609/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 24 14:16:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:22.578262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:22.737870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa648f-b6a9-434a-9045-1801a9252a0c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:22.780491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:22.781058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:22.781444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:22.781763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:22.822641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:22.893334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:22.895028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:23.046495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 605/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:16:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 14:16:23.055071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:23.084120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:23.188369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5c2aa-132d-4c22-83c5-44939fd0986a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:23.238228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:23.239347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:23.239749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:23.240053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:23.253014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:23.254810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:23.275169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:23.277056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:23.279189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:23.279189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:23.279189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:23.282420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 610/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:23.284408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:23.395797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:23.464124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876528-ddb3-4548-9bfb-240072876e8a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:23.518655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:23.519785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:23.520925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:23.521093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:23.522149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:23.522507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:23.524265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:23.539236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 606/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:23.545403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:23.642832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:23.686404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa331c9a-cd07-459f-96fe-ca496bbfd3e0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:23.741274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:23.742103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:23.747149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:23.747661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:23.758901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:23.760548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:16:23.774677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:23.776433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:23.778310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 611/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:23.785371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:23.936877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc165dc-4a9d-41dc-8665-a880082bc287 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:23.971097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:23.987095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad898a-ad81-4afe-bf64-fba118d8aac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:23.988150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad898a-ad81-4afe-bf64-fba118d8aac8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:23.989203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad898a-ad81-4afe-bf64-fba118d8aac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:23.989707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad898a-ad81-4afe-bf64-fba118d8aac8 None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:23.991367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad898a-ad81-4afe-bf64-fba118d8aac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:23.991865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad898a-ad81-4afe-bf64-fba118d8aac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:23.993484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad898a-ad81-4afe-bf64-fba118d8aac8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:24.008972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 607/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:24.016112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:24.162398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9277b-dffb-4383-8859-2ae65cc6421d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:24.197925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:24.204868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a8695-3b16-4b00-8629-c7d604aff84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:24.205498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8695-3b16-4b00-8629-c7d604aff84f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:24.205819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8695-3b16-4b00-8629-c7d604aff84f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:24.206128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8695-3b16-4b00-8629-c7d604aff84f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:24.206872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a8695-3b16-4b00-8629-c7d604aff84f None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=4739aa6ea7c043d999a3280deb9963d0) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:24.207174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a8695-3b16-4b00-8629-c7d604aff84f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:24.209365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a8695-3b16-4b00-8629-c7d604aff84f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:24.215927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 612/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:16:24 2022] GET /identity/v3/role_assignments?user.id=4739aa6ea7c043d999a3280deb9963d0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:26.319663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ad62f-638c-426e-910c-5d3ea2f75a0d None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:26.320475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ad62f-638c-426e-910c-5d3ea2f75a0d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:26.320475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ad62f-638c-426e-910c-5d3ea2f75a0d None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:26.321737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 608/1220] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:16:26.328271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a8ccd-d5b2-4290-9150-76b38027c780 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:26.328830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a8ccd-d5b2-4290-9150-76b38027c780 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:26.328970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a8ccd-d5b2-4290-9150-76b38027c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:26.351723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ec5a8ccd-d5b2-4290-9150-76b38027c780 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:26.353190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ec5a8ccd-d5b2-4290-9150-76b38027c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:26.438661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 613/1221] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:16:26.731639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ad62f-638c-426e-910c-5d3ea2f75a0d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:26.869126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ad62f-638c-426e-910c-5d3ea2f75a0d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:26.911746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:26.912644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:26.912920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:26.913188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:26.925636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:16:26.931154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:26.932468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:26.934241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 609/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:26.938936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a8ccd-d5b2-4290-9150-76b38027c780 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:27.063231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a8ccd-d5b2-4290-9150-76b38027c780 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:27.094118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:27.106250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:27.106996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:27.107452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:27.107933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:27.109131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:27.109612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:27.111281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:27.122375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 614/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:16:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:27.128474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:27.188480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:27.258913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186c476-7947-4e62-bd6b-6a9875df8cf1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:27.299864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:27.300504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:27.300784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:27.301062 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:27.312812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:27.314504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:27.331841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:27.334921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:27.336039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:27.336039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:27.336039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:27.336039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 610/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:27.343148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:27.478888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda45f9-7372-4d57-b048-48feb38359e3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:27.519178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:27.519811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:27.520645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:27.521601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:27.522308 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:27.522616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:27.524130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:27.538671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 615/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:27.545808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:27.654751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:27.661632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf9b3-f1f0-40e7-8809-7d289162b7ec None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:27.699414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:27.700065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:27.700344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:27.700616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:27.709264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:27.716235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:16:27.724887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:27.727245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:27.729098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 611/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:27.732945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:27.791825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:27.866154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac617b-2eb2-428b-8a4e-0c3c6c7c58da None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:27.914576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:27.915523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:27.916017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:27.916497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:27.917963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:27.918572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:27.921469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:27.956702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 616/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 24 14:16:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:27.963226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:28.120496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db809f7-503a-49a0-8293-248217dfe42c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:28.160484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:28.161651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:28.161932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:28.162239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:28.174426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:28.175838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:28.192493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:28.194800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:28.197023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:28.197023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:28.197023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:28.197023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 612/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:28.200146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:28.318533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212178b-2fab-44ee-95d1-405ec84f96f2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:28.321268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:28.361261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:28.362402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:28.362784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:28.363183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:28.364215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:28.365521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:28.367102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:28.381298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 617/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:28.387437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:28.399296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:28.553464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ae40-cb86-4841-a872-cd989e53c63c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:28.608269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:28.609172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:28.609645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:28.610152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:28.618326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:28.619803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:16:28.632752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:28.633812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:28.635614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 613/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:28.640249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:28.760387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6e28-01b4-4238-9b89-49ba0af102ce None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:28.803941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c1f2e-26d6-4768-91f0-b8c19789fac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:28.804468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c1f2e-26d6-4768-91f0-b8c19789fac1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:28.804742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c1f2e-26d6-4768-91f0-b8c19789fac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:28.805009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c1f2e-26d6-4768-91f0-b8c19789fac1 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:28.805726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c1f2e-26d6-4768-91f0-b8c19789fac1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:28.806055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c1f2e-26d6-4768-91f0-b8c19789fac1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:28.807528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c1f2e-26d6-4768-91f0-b8c19789fac1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:28.822750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 618/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:28.833941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:28.935891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:28.977857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856d23-3d55-4ada-85ac-8771aefd67c3 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:29.015326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07c1a-259d-490e-865e-ad619c8ef4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:29.016107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07c1a-259d-490e-865e-ad619c8ef4df None admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:29.016495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07c1a-259d-490e-865e-ad619c8ef4df None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:29.016882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07c1a-259d-490e-865e-ad619c8ef4df None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/4739aa6ea7c043d999a3280deb9963d0/roles/adf17858daf74c2ea9cc93775172b567` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:29.036885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a07c1a-259d-490e-865e-ad619c8ef4df None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=4739aa6ea7c043d999a3280deb9963d0, role_id=adf17858daf74c2ea9cc93775172b567)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:29.038584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a07c1a-259d-490e-865e-ad619c8ef4df None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:29.243599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 614/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:16:28 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/4739aa6ea7c043d999a3280deb9963d0/roles/adf17858daf74c2ea9cc93775172b567 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:16:31.341444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e43f-2fa3-42cd-990b-1171c1836856 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:31.342119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e43f-2fa3-42cd-990b-1171c1836856 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:31.342562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e43f-2fa3-42cd-990b-1171c1836856 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:31.345183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 619/1233] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:16:31.352689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd5430-8cfc-486a-b251-59e09caee450 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:31.353546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd5430-8cfc-486a-b251-59e09caee450 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:31.353737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd5430-8cfc-486a-b251-59e09caee450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:31.377052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-69fd5430-8cfc-486a-b251-59e09caee450 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:31.378371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-69fd5430-8cfc-486a-b251-59e09caee450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:31.458431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 615/1234] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:16:31.736918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e43f-2fa3-42cd-990b-1171c1836856 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:31.767306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:31.865405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e43f-2fa3-42cd-990b-1171c1836856 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:31.907712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:31.908403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:31.912914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:31.912914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:31.922130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:16:31.928287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:31.930017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:31.931903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 620/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 24 14:16:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:31.938323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd5430-8cfc-486a-b251-59e09caee450 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:31.994939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:32.059875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd5430-8cfc-486a-b251-59e09caee450 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:32.098303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:32.099079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:32.099361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:32.099630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:32.100377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] RBAC: Adding query filter params (name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:32.100706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:32.102373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:32.111746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 616/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 24 14:16:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:32.118943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:32.226036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:32.246463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7036e06-ebf8-4152-9201-a487551b44bb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:32.288868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:32.299700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:32.299700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:32.299700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:32.303173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:32.304850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:32.322748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:32.324443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:32.326549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:32.326549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:32.326549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:32.326549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 621/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:32.331831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:32.455497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cef57f-990a-44bb-930e-05baac7a489a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:32.466813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:32.496792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:32.497330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:32.497606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:32.497875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:32.498565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:32.498868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:32.500277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:32.517019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 617/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:32.524064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:32.642244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6f9e1-0409-4be4-a33f-f3aab35bd9d5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:32.682968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:32.683668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:32.683976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:32.684282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:32.687787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:32.693906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:32.695396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:16:32.708530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:32.710403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:32.712510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 622/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:32.717660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:32.832114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec9484-d763-473f-9b00-0549a754fd34 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:32.871327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:32.871949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:32.872226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:32.872515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:32.873433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:32.873614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:32.875096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:32.895679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 618/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 24 14:16:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:32.902208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:33.018444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ada7bc-0fac-4c7b-94e0-252d27b57233 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:33.060626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:33.061257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:33.061535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:33.061805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:33.133475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:33.140593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:33.142096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:33.257301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 623/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:16:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 14:16:33.264684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:33.297358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:33.383461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be378df-5665-4bf6-8f73-196a0269bed4 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:33.423299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:33.423900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:33.424179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:33.424452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:33.436350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:33.437820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:33.454487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:33.456962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:33.459731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:33.459731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:33.459731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:33.459731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 619/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:33.463245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:33.637920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:33.644519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29646968-1e4d-4a63-a6d9-87fee4d5c69d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:33.682402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:33.682932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:33.683205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:33.683473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:33.684189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:33.684545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:33.685957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:33.703152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 624/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:33.710170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:33.799852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:33.824640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776cf64-0dd3-48a2-b18f-75b0bb64a864 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:33.862181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:33.862842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:33.863121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:33.863389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:33.871135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:33.872832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:16:33.885436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:33.886666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:33.887593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 620/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:33.893001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:34.080634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdec67-49dd-4900-90cc-438a7d9f672c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:34.138114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a62b8-1b92-4b35-abb2-eb7d8535c101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:34.138740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a62b8-1b92-4b35-abb2-eb7d8535c101 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:34.139011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a62b8-1b92-4b35-abb2-eb7d8535c101 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:34.139280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a62b8-1b92-4b35-abb2-eb7d8535c101 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:34.139997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a62b8-1b92-4b35-abb2-eb7d8535c101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:34.140298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a62b8-1b92-4b35-abb2-eb7d8535c101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:34.141782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a62b8-1b92-4b35-abb2-eb7d8535c101 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:34.156567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 625/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:34.167853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:34.331551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54f483-1081-45ee-9fae-d4ace568728c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:34.399456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479b9d8-81bf-46b0-8313-099e4ac7254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:34.400398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479b9d8-81bf-46b0-8313-099e4ac7254f None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:34.400845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479b9d8-81bf-46b0-8313-099e4ac7254f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:34.401307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479b9d8-81bf-46b0-8313-099e4ac7254f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:34.402386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479b9d8-81bf-46b0-8313-099e4ac7254f None admin] RBAC: Adding query filter params (role.id=adf17858daf74c2ea9cc93775172b567, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=4739aa6ea7c043d999a3280deb9963d0) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:34.402705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479b9d8-81bf-46b0-8313-099e4ac7254f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:34.404548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479b9d8-81bf-46b0-8313-099e4ac7254f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:34.412259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 621/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:16:34 2022] GET /identity/v3/role_assignments?user.id=4739aa6ea7c043d999a3280deb9963d0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=adf17858daf74c2ea9cc93775172b567 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:36.533865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc0fa7-f168-440a-ba60-71b4c328e21d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:36.534616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc0fa7-f168-440a-ba60-71b4c328e21d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:36.534616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc0fa7-f168-440a-ba60-71b4c328e21d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:36.535905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 626/1247] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:16:36.542192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab825b-967f-473e-9b0f-c15f58ab145c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:36.542730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab825b-967f-473e-9b0f-c15f58ab145c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:36.542842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab825b-967f-473e-9b0f-c15f58ab145c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:36.549634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:36.571669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-aeab825b-967f-473e-9b0f-c15f58ab145c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:36.573258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aeab825b-967f-473e-9b0f-c15f58ab145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:36.648650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 622/1248] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:16:36.922963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc0fa7-f168-440a-ba60-71b4c328e21d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:36.944458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:37.044446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc0fa7-f168-440a-ba60-71b4c328e21d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:37.085633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:37.086379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:37.086770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:37.087088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:37.102983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:16:37.111769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:37.113011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:37.114742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 627/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:37.124560 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab825b-967f-473e-9b0f-c15f58ab145c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:37.300183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab825b-967f-473e-9b0f-c15f58ab145c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:37.337002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:37.341835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:37.342506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:37.342783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:37.343050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:37.343736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:37.344037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:37.345442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:37.355434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 623/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:37.361516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:37.472230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:37.499156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e7392-676e-4b4b-b52d-5109592598fb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:37.538475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:37.539164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:37.539387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:37.539653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:37.552917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:37.554362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:37.570995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:37.572228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:37.573150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:37.573150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:37.573150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:37.573150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 628/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:37.579065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:37.699813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fb24b-3584-441a-b0e7-17bf8d4a9590 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:37.740262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:37.740897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:37.741243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:37.741523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:37.742299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:37.742606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:37.744032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:37.758881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 624/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:37.765199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:37.888113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351237f8-cd39-4988-8ca9-b836afe85097 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:37.923114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:37.932887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:37.933558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:37.933836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:37.934137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:37.943300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:37.947091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:16:37.957969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:37.959748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:37.961398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 629/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:37.967501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:38.035372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:38.091202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303f49-3a17-45bb-92e6-0cd90fc36d5a None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:38.150023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:38.150998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:38.151469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:38.151940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:38.153388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:38.154229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:38.156921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:38.192943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 625/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 24 14:16:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:38.199114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:38.319673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7641e-3c67-4ef3-9c28-92cf9c180f0a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:38.358095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:38.358721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:38.359000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:38.359272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:38.430247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:38.446653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:38.448116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:38.564932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 630/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:16:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 14:16:38.572209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:38.692862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4ff2-901a-4dcf-a35e-fa3e426ac66d None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:38.709001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:38.734245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:38.734905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:38.735181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:38.735452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:38.747844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:38.749351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:38.766859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:38.768687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:38.769630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:38.769630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:38.769630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:38.769630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 626/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:38.775829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:38.932968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:38.955501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe95c8-590b-4685-b720-9562378565ef None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:39.002397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:39.003156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:39.003475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:39.003789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:39.005352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:39.005778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:39.007309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:39.020519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 631/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:39.029859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:39.150439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043eecb6-ff84-4b3d-b552-e3aa4ef27989 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:39.192366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:39.193697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:39.193978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:39.194287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:39.205193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:39.206831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:16:39.220983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:39.222854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:39.224902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 627/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:16:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:39.229310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:39.346876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5b09c-def8-4ea9-aa60-9780505888b3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:39.387674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a99b9-a16f-4b86-8f04-939d83ccd99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:39.388438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a99b9-a16f-4b86-8f04-939d83ccd99b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:39.388821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a99b9-a16f-4b86-8f04-939d83ccd99b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:39.389194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a99b9-a16f-4b86-8f04-939d83ccd99b None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:39.390031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a99b9-a16f-4b86-8f04-939d83ccd99b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:39.390445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a99b9-a16f-4b86-8f04-939d83ccd99b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:39.392024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a99b9-a16f-4b86-8f04-939d83ccd99b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:39.408911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 632/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:16:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:39.415699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:39.446189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:39.558792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1215c7-05bd-47e1-a1e9-26b26adf5018 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:39.618954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd841db-3a6e-4fb1-968c-e2410daaa7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:39.619970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd841db-3a6e-4fb1-968c-e2410daaa7df None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:39.620470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd841db-3a6e-4fb1-968c-e2410daaa7df None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:39.620916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd841db-3a6e-4fb1-968c-e2410daaa7df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:39.622287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd841db-3a6e-4fb1-968c-e2410daaa7df None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=4739aa6ea7c043d999a3280deb9963d0) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:39.622839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd841db-3a6e-4fb1-968c-e2410daaa7df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:39.625519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd841db-3a6e-4fb1-968c-e2410daaa7df None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:39.636529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 628/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:16:39 2022] GET /identity/v3/role_assignments?user.id=4739aa6ea7c043d999a3280deb9963d0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:41.668566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec934e-0808-41f1-9ac6-6f6f75b320d8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:41.669164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec934e-0808-41f1-9ac6-6f6f75b320d8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:41.669164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec934e-0808-41f1-9ac6-6f6f75b320d8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:41.670450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 633/1261] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:16:41.677073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0caba6-107e-4416-8ed8-94185afa9d43 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:41.677731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0caba6-107e-4416-8ed8-94185afa9d43 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:41.677864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0caba6-107e-4416-8ed8-94185afa9d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:41.697049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4c0caba6-107e-4416-8ed8-94185afa9d43 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:41.698348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4c0caba6-107e-4416-8ed8-94185afa9d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:41.779274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 629/1262] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:16:42.057502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec934e-0808-41f1-9ac6-6f6f75b320d8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:42.070888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:42.184860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec934e-0808-41f1-9ac6-6f6f75b320d8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:42.226671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:42.227473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:42.227751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:42.228028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:42.240713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:16:42.245613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:42.247471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:42.249119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 634/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:16:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:42.258568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0caba6-107e-4416-8ed8-94185afa9d43 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:42.294619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:42.412153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0caba6-107e-4416-8ed8-94185afa9d43 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:42.452254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:42.452876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:42.453155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:42.453430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:42.454177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:42.454484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:42.455909 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:42.466141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 630/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:16:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:42.474293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:42.559563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:42.599451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f90488-c78c-4018-847a-b704e7bdc28f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:42.639433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:42.640077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:42.640362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:42.640733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:42.653259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:42.655186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:42.672335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:42.674597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:42.675864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:42.675864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:42.675864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:42.675864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 635/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:42.681481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:42.776248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:42.797851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff743e-3184-4a97-9ad8-6bfab26e4e39 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:42.838424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:42.839124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:42.839478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:42.839835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:42.840582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:42.840944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:42.842541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:42.856362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 631/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:42.865944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:42.988911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e3967-eba3-4a7e-90af-20ea9593d69a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:43.012522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:43.030731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:43.031388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:43.031665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:43.031937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:43.041575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:43.043149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:16:43.055874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:43.057856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:43.059793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 636/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:43.063413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:43.178561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc15f79-9b46-490d-bece-a992a14f85bf None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:43.218136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:43.218927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:43.219304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:43.219677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:43.220607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:43.221092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:43.222858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:43.235341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:43.248830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 632/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 24 14:16:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:43.255956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:43.384895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c38692-e2e4-4b8d-8d7a-dceda69af5c0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:43.431855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:43.432602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:43.432884 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:43.433159 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:43.446494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:43.448033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:43.465788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:43.471826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:43.474423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:43.474423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:43.474423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:43.474423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 637/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:43.476318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:43.602648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b207d-99d7-44f0-a4a7-25d303f2d3e8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:43.644983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:43.645782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:43.646301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:43.656630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:43.656630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:43.656630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:43.656630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:43.663153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 633/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:43.669184 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:43.707209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:43.785294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bac62-5343-479b-a675-333f3ebc08e1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:43.826187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:43.828223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:43.828223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:43.828223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:43.835487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:43.837018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:16:43.850319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:43.864822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:43.866448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 638/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:43.867523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:43.928179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:44.001370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab08a-ee61-42b9-ad0b-4395b2e0fad8 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:44.068459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3037e3-4a63-403b-8061-52e27b745dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:44.069514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3037e3-4a63-403b-8061-52e27b745dbf None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:44.070026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3037e3-4a63-403b-8061-52e27b745dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:44.070502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3037e3-4a63-403b-8061-52e27b745dbf None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:44.071856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3037e3-4a63-403b-8061-52e27b745dbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:44.072501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3037e3-4a63-403b-8061-52e27b745dbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:44.075219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3037e3-4a63-403b-8061-52e27b745dbf None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:44.099367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 634/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:44.105861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:44.222338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:44.233741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2618db-669a-493a-b6be-75274b0cf8fc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:44.283937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de792c-6178-4fb1-a740-15e3095e2b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:44.284686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de792c-6178-4fb1-a740-15e3095e2b8e None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:44.285041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de792c-6178-4fb1-a740-15e3095e2b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:44.285400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de792c-6178-4fb1-a740-15e3095e2b8e None admin] PATH_INFO: `/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/4739aa6ea7c043d999a3280deb9963d0/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:44.287772 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de792c-6178-4fb1-a740-15e3095e2b8e None admin] RBAC: Authorizing `identity:create_grant(project_id=f4f2fa9f48864e54b67f4c732710b225, user_id=4739aa6ea7c043d999a3280deb9963d0, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:44.289292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de792c-6178-4fb1-a740-15e3095e2b8e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:44.379167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 639/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 24 14:16:44 2022] PUT /identity/v3/projects/f4f2fa9f48864e54b67f4c732710b225/users/4739aa6ea7c043d999a3280deb9963d0/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:16:46.372039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaf63d-1c20-4a6d-8c8e-b21b567588ac None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:46.372641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaf63d-1c20-4a6d-8c8e-b21b567588ac None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:46.372641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaf63d-1c20-4a6d-8c8e-b21b567588ac None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:46.373934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 635/1274] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:16:46.380670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ff36f-acfe-47dc-84f6-267f33882fe8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:46.381244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ff36f-acfe-47dc-84f6-267f33882fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:46.381244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ff36f-acfe-47dc-84f6-267f33882fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:46.401137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-595ff36f-acfe-47dc-84f6-267f33882fe8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:46.402431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-595ff36f-acfe-47dc-84f6-267f33882fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:46.485689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 640/1275] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:16:46.763487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaf63d-1c20-4a6d-8c8e-b21b567588ac None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:46.879801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaf63d-1c20-4a6d-8c8e-b21b567588ac None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:46.913379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:46.945367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294866-f230-4d05-9878-047b56ca272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:46.946493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294866-f230-4d05-9878-047b56ca272c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:46.947018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294866-f230-4d05-9878-047b56ca272c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:46.947525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294866-f230-4d05-9878-047b56ca272c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:46.967087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38294866-f230-4d05-9878-047b56ca272c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-38294866-f230-4d05-9878-047b56ca272c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 14:16:46.977714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:46.979553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:46.981539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 636/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 24 14:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:46.987237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ff36f-acfe-47dc-84f6-267f33882fe8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:47.073547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:47.101408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ff36f-acfe-47dc-84f6-267f33882fe8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:47.161820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:47.162903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:47.163398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:47.163861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:47.165104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:47.165632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:47.168256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:47.185070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 641/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 24 14:16:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:47.190887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294866-f230-4d05-9878-047b56ca272c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:47.313399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294866-f230-4d05-9878-047b56ca272c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:47.366767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:47.367467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:47.367789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:47.368175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:47.384331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:47.385905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:47.402822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:47.404915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:47.406679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:47.406679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:47.406679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:47.406679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 637/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:47.414309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:47.560429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c056a-9751-4834-8b8b-e1270fdc5eac None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:47.624590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:47.625561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:47.626093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:47.626768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:47.627965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:47.628522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:47.631242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:47.656197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 642/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:47.666227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:47.702424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:47.813569 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a967ec3-3bfc-4571-a5a4-6e0d5fb2ebc5 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:47.856487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:47.857174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:47.857480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:47.857755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:47.871122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:47.873386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 14:16:47.894466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:47.896231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 14:16:47.897205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:47.898262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 638/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:47.908540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:47.916084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:48.026013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a6ef-9949-464d-afb1-b6b113eba087 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:48.068866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:48.069495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:48.069777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:48.070119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:48.070916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:48.071219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:48.072684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:48.094493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 643/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 24 14:16:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:48.102554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:48.207874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:48.220168 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812d88-86e6-436e-9806-22b6265be4ae None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:48.284495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:48.285709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:48.286242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:48.286727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:48.420307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:48.423248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:48.561851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:48.573735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 639/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 24 14:16:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 14:16:48.580333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:48.653551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:48.697090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6ea4d-bafe-4a22-b865-7d853ef16f1c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:48.737380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:48.738059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:48.738365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:48.738659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:48.750682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:48.752385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 14:16:48.768900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:48.771319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 14:16:48.772522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 14:16:48.772522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 14:16:48.772522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:48.772522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 644/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:48.777065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:48.895817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a1d2f-3f60-405e-8d15-a2c117da5914 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:48.946837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:48.947914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:48.948426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:48.948929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:48.950159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:48.950711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:48.953567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:48.979928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 640/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 24 14:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:48.985958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:49.096698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bd0fc-1de1-4121-a071-d521278bdc0d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:49.113491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:49.136124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:49.136808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:49.137097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:49.137375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:49.145084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:49.146661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 14:16:49.160376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:49.161945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 14:16:49.162965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 14:16:49.163442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 645/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 24 14:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:49.170609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:49.264607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:49.354733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c50e1-9741-41f1-b998-6dae3319118c None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:49.396240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde2729-bd05-4497-a461-96c67930fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:49.397896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2729-bd05-4497-a461-96c67930fcfb None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:49.398139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2729-bd05-4497-a461-96c67930fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:49.398471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2729-bd05-4497-a461-96c67930fcfb None admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:49.399557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde2729-bd05-4497-a461-96c67930fcfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:49.400099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde2729-bd05-4497-a461-96c67930fcfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:49.402224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde2729-bd05-4497-a461-96c67930fcfb None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:49.417547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 641/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 24 14:16:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:49.424227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:49.539882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2d38d-64d2-46f6-bad4-bdd39782498b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:49.580706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c2495-3fb2-4118-87e7-d09a4343914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:49.581740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2495-3fb2-4118-87e7-d09a4343914a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:49.581740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2495-3fb2-4118-87e7-d09a4343914a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:49.582102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2495-3fb2-4118-87e7-d09a4343914a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:49.583974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c2495-3fb2-4118-87e7-d09a4343914a None admin] RBAC: Adding query filter params (role.id=6dbae174ede84d6f9b9feaa4579bab05, scope.project.id=f4f2fa9f48864e54b67f4c732710b225, user.id=4739aa6ea7c043d999a3280deb9963d0) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:49.584306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c2495-3fb2-4118-87e7-d09a4343914a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:49.585773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c2495-3fb2-4118-87e7-d09a4343914a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:49.592465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 646/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 24 14:16:49 2022] GET /identity/v3/role_assignments?user.id=4739aa6ea7c043d999a3280deb9963d0&scope.project.id=f4f2fa9f48864e54b67f4c732710b225&role.id=6dbae174ede84d6f9b9feaa4579bab05 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:51.740259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b31db-f518-431c-8bc0-f0739acf2cdb None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:51.741047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b31db-f518-431c-8bc0-f0739acf2cdb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:51.741047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b31db-f518-431c-8bc0-f0739acf2cdb None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:51.742098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 642/1288] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:16:51.748656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60ed5-dd7c-493a-8508-4172469d2b66 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:51.749268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60ed5-dd7c-493a-8508-4172469d2b66 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:51.749268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60ed5-dd7c-493a-8508-4172469d2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:51.770725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-71a60ed5-dd7c-493a-8508-4172469d2b66 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:51.772166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-71a60ed5-dd7c-493a-8508-4172469d2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:51.809617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:51.871019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 647/1289] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:16:52.162672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b31db-f518-431c-8bc0-f0739acf2cdb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:52.285282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b31db-f518-431c-8bc0-f0739acf2cdb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:52.299186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:52.326399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:52.327042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:52.327315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:52.327580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:52.328207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:52.329599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:52.339522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:52.341001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:52.342319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 643/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:16:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 14:16:52.346874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a60ed5-dd7c-493a-8508-4172469d2b66 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:52.461670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a60ed5-dd7c-493a-8508-4172469d2b66 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:52.504369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e300e0-7649-4061-90fa-9ff18da72eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:52.505162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300e0-7649-4061-90fa-9ff18da72eec None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:52.505444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300e0-7649-4061-90fa-9ff18da72eec None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:52.505720 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300e0-7649-4061-90fa-9ff18da72eec None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:52.506390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e300e0-7649-4061-90fa-9ff18da72eec None admin] RBAC: Adding query filter params (name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:52.506689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e300e0-7649-4061-90fa-9ff18da72eec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:52.508058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e300e0-7649-4061-90fa-9ff18da72eec None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:52.513383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 648/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 24 14:16:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:52.519406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:52.632801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd89ed5-937f-4614-8bec-fd697bbe7ec1 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:52.669580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235ceb6-1f03-47e0-9197-582ef6557c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:52.670230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ceb6-1f03-47e0-9197-582ef6557c8a None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:52.670538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ceb6-1f03-47e0-9197-582ef6557c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:52.670766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ceb6-1f03-47e0-9197-582ef6557c8a None admin] PATH_INFO: `/v3/services` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:52.671405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ceb6-1f03-47e0-9197-582ef6557c8a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:52.671799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ceb6-1f03-47e0-9197-582ef6557c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:52.673087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ceb6-1f03-47e0-9197-582ef6557c8a None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:52.679123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 644/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 24 14:16:52 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:54.770729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002042c-938e-47e5-b192-7e9789b5e2c1 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:54.771573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002042c-938e-47e5-b192-7e9789b5e2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:54.771573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002042c-938e-47e5-b192-7e9789b5e2c1 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:54.772790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 649/1293] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:16:54.779329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b2c06-1a0e-4aab-aaab-c133981e48e4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:54.779958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b2c06-1a0e-4aab-aaab-c133981e48e4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:54.780214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b2c06-1a0e-4aab-aaab-c133981e48e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:54.800583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-cd4b2c06-1a0e-4aab-aaab-c133981e48e4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:54.802849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cd4b2c06-1a0e-4aab-aaab-c133981e48e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:54.840392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:54.886520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 645/1294] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:16:55.174688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002042c-938e-47e5-b192-7e9789b5e2c1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:55.219963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:55.306370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002042c-938e-47e5-b192-7e9789b5e2c1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:55.347227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce78b5-ef12-4eee-8991-24aff0f1e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:55.347685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce78b5-ef12-4eee-8991-24aff0f1e214 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:55.348605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce78b5-ef12-4eee-8991-24aff0f1e214 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:55.349636 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce78b5-ef12-4eee-8991-24aff0f1e214 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:55.350364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce78b5-ef12-4eee-8991-24aff0f1e214 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:55.351777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce78b5-ef12-4eee-8991-24aff0f1e214 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:55.453081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 650/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 24 14:16:55 2022] POST /identity/v3/services => generated 246 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:16:57.583011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1072-b1ed-4f78-a5c7-0da215d829fb None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:57.583905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1072-b1ed-4f78-a5c7-0da215d829fb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:57.583905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1072-b1ed-4f78-a5c7-0da215d829fb None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:57.585571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 646/1296] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:16:57.593796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b57cd-e452-4367-abbc-0066122690ab None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:57.599529 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b57cd-e452-4367-abbc-0066122690ab None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:57.599529 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b57cd-e452-4367-abbc-0066122690ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:57.626267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2e7b57cd-e452-4367-abbc-0066122690ab None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:16:57.628676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2e7b57cd-e452-4367-abbc-0066122690ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:16:57.727603 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 651/1297] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:16:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 14:16:58.023803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1072-b1ed-4f78-a5c7-0da215d829fb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:58.171452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b1072-b1ed-4f78-a5c7-0da215d829fb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:58.228943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:58.229949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:58.231215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:58.231215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:58.231960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:58.234377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:16:58.246751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:16:58.247844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:16:58.248682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 647/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:16:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:16:58.256954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b57cd-e452-4367-abbc-0066122690ab None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:58.289056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:58.409322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b57cd-e452-4367-abbc-0066122690ab None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:58.477897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:58.478677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:58.478953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:58.479236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:58.479893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:58.480202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:58.481599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:58.487759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 652/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 24 14:16:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:16:58.495350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:58.532022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:58.619254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a27e7f-433f-4b89-871a-7b49bc638298 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:58.659482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad474d5-658e-4075-ba35-925475eabf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:58.660167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad474d5-658e-4075-ba35-925475eabf4d None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:58.661091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad474d5-658e-4075-ba35-925475eabf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:58.661571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad474d5-658e-4075-ba35-925475eabf4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:58.662681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad474d5-658e-4075-ba35-925475eabf4d None admin] RBAC: Adding query filter params (interface=public, service_id=6f20f20acf2444f39ca24de4860935b5, region_id=RegionOne) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:16:58.663041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad474d5-658e-4075-ba35-925475eabf4d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:58.664635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad474d5-658e-4075-ba35-925475eabf4d None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:58.670696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 648/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 24 14:16:58 2022] GET /identity/v3/endpoints?service_id=6f20f20acf2444f39ca24de4860935b5&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:16:58.677774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:16:58.771502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:16:58.793205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72102bd1-5abc-45b1-8070-104e1dbdd057 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:16:58.832305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2a5bf-e366-4ce1-9359-4525feeeb500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:16:58.833068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2a5bf-e366-4ce1-9359-4525feeeb500 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:16:58.833478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2a5bf-e366-4ce1-9359-4525feeeb500 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:16:58.833840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2a5bf-e366-4ce1-9359-4525feeeb500 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:16:58.834643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2a5bf-e366-4ce1-9359-4525feeeb500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:16:58.836167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2a5bf-e366-4ce1-9359-4525feeeb500 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:16:58.843774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 653/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 24 14:16:58 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:17:00.882097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee580012-71f6-49ee-8f22-edb535b28b38 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:17:00.882716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee580012-71f6-49ee-8f22-edb535b28b38 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:17:00.882979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee580012-71f6-49ee-8f22-edb535b28b38 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:17:00.885485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 649/1302] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:17:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:17:00.892016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5ee9-bdb1-4ec5-9aa8-dec31015bcae None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:17:00.892599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5ee9-bdb1-4ec5-9aa8-dec31015bcae None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:17:00.892599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5ee9-bdb1-4ec5-9aa8-dec31015bcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:17:00.917333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d07e5ee9-bdb1-4ec5-9aa8-dec31015bcae None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:17:00.918652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d07e5ee9-bdb1-4ec5-9aa8-dec31015bcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:17:00.998996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 654/1303] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:17:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 14:17:01.276709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee580012-71f6-49ee-8f22-edb535b28b38 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:17:01.399234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:17:01.407287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee580012-71f6-49ee-8f22-edb535b28b38 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:17:01.449754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:17:01.450498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:17:01.450824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:17:01.451140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:17:01.451804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:17:01.453465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 14:17:01.465843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 14:17:01.470496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application with Lock( Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self._enter() Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application return self.creator() Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application created_value = creator( Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: ERROR keystone.server.flask.application Aug 24 14:17:01.472127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 650/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 24 14:17:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 14:17:01.476267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5ee9-bdb1-4ec5-9aa8-dec31015bcae None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:17:01.586452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:17:01.614553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5ee9-bdb1-4ec5-9aa8-dec31015bcae None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:17:01.655880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771121a6-153e-4689-9f93-580021949e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:17:01.656905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771121a6-153e-4689-9f93-580021949e71 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:17:01.657476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771121a6-153e-4689-9f93-580021949e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:17:01.657903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771121a6-153e-4689-9f93-580021949e71 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:17:01.659155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771121a6-153e-4689-9f93-580021949e71 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:17:01.659506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771121a6-153e-4689-9f93-580021949e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:17:01.661634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771121a6-153e-4689-9f93-580021949e71 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:17:01.668181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 655/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 24 14:17:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:17:01.674598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:17:01.796920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b794-1a53-4d3f-b4bb-ef4bca42b913 None admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:17:01.838537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd4184-c44c-4379-b119-5e452533415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:17:01.839066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4184-c44c-4379-b119-5e452533415f None admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:17:01.839348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4184-c44c-4379-b119-5e452533415f None admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:17:01.839621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd4184-c44c-4379-b119-5e452533415f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:17:01.840334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd4184-c44c-4379-b119-5e452533415f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:17:01.841763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd4184-c44c-4379-b119-5e452533415f None admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:17:02.029891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 651/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:17:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:17:11.664589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=30 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:17:12.024520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:33.368243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b342-440f-4a4b-a4c3-f390299cff5b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:33.369665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b342-440f-4a4b-a4c3-f390299cff5b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:33.370140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b342-440f-4a4b-a4c3-f390299cff5b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:33.372101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 656/1307] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:25:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:33.379524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623ac3-87d4-46f7-8df3-6a74013e9e4c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:33.380542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623ac3-87d4-46f7-8df3-6a74013e9e4c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:33.381070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51623ac3-87d4-46f7-8df3-6a74013e9e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:33.525929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-51623ac3-87d4-46f7-8df3-6a74013e9e4c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:33.527333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-51623ac3-87d4-46f7-8df3-6a74013e9e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:33.723981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 652/1308] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:34.051003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46b7a5-95d2-4ef2-a3d6-248bcdff7a96 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:34.051262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46b7a5-95d2-4ef2-a3d6-248bcdff7a96 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:34.051499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46b7a5-95d2-4ef2-a3d6-248bcdff7a96 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:34.053536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 657/1309] 10.222.0.49 () {54 vars in 965 bytes} [Wed Aug 24 14:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:34.063834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa931648-f975-4339-976f-dcdaec9b61db None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:34.064614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa931648-f975-4339-976f-dcdaec9b61db None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:34.065235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa931648-f975-4339-976f-dcdaec9b61db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:34.246446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-aa931648-f975-4339-976f-dcdaec9b61db None None] MFA Rules not processed for user `48364ccf19434c4c9c09b4e5ce4bfc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:34.249590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aa931648-f975-4339-976f-dcdaec9b61db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:34.300432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:34.446854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 653/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Wed Aug 24 14:25:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:34.455852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46b7a5-95d2-4ef2-a3d6-248bcdff7a96 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:25:34.458350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6a46b7a5-95d2-4ef2-a3d6-248bcdff7a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:34.599343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46b7a5-95d2-4ef2-a3d6-248bcdff7a96 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:25:34.727653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20568e-3f4b-475f-be0b-f5bf1ce2c7bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:25:34.728444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20568e-3f4b-475f-be0b-f5bf1ce2c7bb service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:34.728987 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20568e-3f4b-475f-be0b-f5bf1ce2c7bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:34.729328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20568e-3f4b-475f-be0b-f5bf1ce2c7bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:34.731498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3c20568e-3f4b-475f-be0b-f5bf1ce2c7bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:34.818288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:34.906826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20568e-3f4b-475f-be0b-f5bf1ce2c7bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:25:34.908611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20568e-3f4b-475f-be0b-f5bf1ce2c7bb service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:25:34.933898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 658/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:25:37.544289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7db0a-e613-4cf4-ae3a-30b0f4e00c46 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:37.544854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7db0a-e613-4cf4-ae3a-30b0f4e00c46 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:37.544854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7db0a-e613-4cf4-ae3a-30b0f4e00c46 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:37.546249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 654/1312] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:37.552990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c30f-4232-451e-aa30-4664f34abbbe None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:37.553590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c30f-4232-451e-aa30-4664f34abbbe None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:37.553590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c30f-4232-451e-aa30-4664f34abbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:37.657247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c9a9c30f-4232-451e-aa30-4664f34abbbe None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:37.658556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c9a9c30f-4232-451e-aa30-4664f34abbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:37.769984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 659/1313] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:38.066994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7db0a-e613-4cf4-ae3a-30b0f4e00c46 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:25:38.068532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4ec7db0a-e613-4cf4-ae3a-30b0f4e00c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:38.192933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7db0a-e613-4cf4-ae3a-30b0f4e00c46 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:25:38.291833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:38.295402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19fce-f051-4725-a7d1-1d6ee12f0f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:25:38.296079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19fce-f051-4725-a7d1-1d6ee12f0f9a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:38.296397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19fce-f051-4725-a7d1-1d6ee12f0f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:38.296711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19fce-f051-4725-a7d1-1d6ee12f0f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:38.298684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-51b19fce-f051-4725-a7d1-1d6ee12f0f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:38.430468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19fce-f051-4725-a7d1-1d6ee12f0f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:25:38.431956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19fce-f051-4725-a7d1-1d6ee12f0f9a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:25:38.454540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 655/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:25:41.154613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59473-8090-4d05-a79e-6c6a1b020304 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:41.155169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59473-8090-4d05-a79e-6c6a1b020304 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:41.155169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59473-8090-4d05-a79e-6c6a1b020304 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:41.156626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 660/1315] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:41.163133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb646bf-6c72-4ae1-bf94-03dab1d7a9c0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:41.164203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb646bf-6c72-4ae1-bf94-03dab1d7a9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:41.164532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb646bf-6c72-4ae1-bf94-03dab1d7a9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:41.242769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6fb646bf-6c72-4ae1-bf94-03dab1d7a9c0 None None] MFA Rules not processed for user `7f091d543cf04f68b41e4456077bc426`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:41.244010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6fb646bf-6c72-4ae1-bf94-03dab1d7a9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:41.302779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:41.356776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 656/1316] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:25:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:41.651057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc59473-8090-4d05-a79e-6c6a1b020304 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:25:41.652607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-afc59473-8090-4d05-a79e-6c6a1b020304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:41.662880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:41.811787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc59473-8090-4d05-a79e-6c6a1b020304 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:25:41.890932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba7a17-9afc-4e40-a871-94d651376434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:25:41.891545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba7a17-9afc-4e40-a871-94d651376434 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:41.891819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba7a17-9afc-4e40-a871-94d651376434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:41.892084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba7a17-9afc-4e40-a871-94d651376434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:41.893717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a4ba7a17-9afc-4e40-a871-94d651376434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:41.966652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:42.031140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba7a17-9afc-4e40-a871-94d651376434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:25:42.032815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba7a17-9afc-4e40-a871-94d651376434 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:25:42.054712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 661/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:25:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:25:44.966753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a25217-7fd1-489f-bce7-2f909b2738c8 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:44.967348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a25217-7fd1-489f-bce7-2f909b2738c8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:44.967348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a25217-7fd1-489f-bce7-2f909b2738c8 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:44.968861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 657/1318] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:44.975088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb827fc-6d35-4297-a455-fb66cc068afa None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:44.976018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb827fc-6d35-4297-a455-fb66cc068afa None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:44.976018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb827fc-6d35-4297-a455-fb66cc068afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:45.060256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9cb827fc-6d35-4297-a455-fb66cc068afa None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:45.061535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9cb827fc-6d35-4297-a455-fb66cc068afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:45.182571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 662/1319] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:25:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:45.482650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a25217-7fd1-489f-bce7-2f909b2738c8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:25:45.484388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-42a25217-7fd1-489f-bce7-2f909b2738c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:45.643070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a25217-7fd1-489f-bce7-2f909b2738c8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:25:45.727621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbace90-e565-4290-bd7d-fb9ab00c57c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:25:45.728263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbace90-e565-4290-bd7d-fb9ab00c57c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:45.728534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbace90-e565-4290-bd7d-fb9ab00c57c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:45.728803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbace90-e565-4290-bd7d-fb9ab00c57c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:45.730603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-edbace90-e565-4290-bd7d-fb9ab00c57c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:45.739156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:45.853665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbace90-e565-4290-bd7d-fb9ab00c57c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:25:45.855164 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbace90-e565-4290-bd7d-fb9ab00c57c7 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:25:45.877546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 658/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:25:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:25:49.308812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3b6ba-1b69-400a-9a8e-c72afc64b0e8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:49.309504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3b6ba-1b69-400a-9a8e-c72afc64b0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:49.309890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3b6ba-1b69-400a-9a8e-c72afc64b0e8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:49.314049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 663/1321] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:25:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:49.323132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3ba73-e59a-4e02-bdff-82d9d97cf980 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:49.325074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3ba73-e59a-4e02-bdff-82d9d97cf980 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:49.325375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3ba73-e59a-4e02-bdff-82d9d97cf980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:49.432558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b3b3ba73-e59a-4e02-bdff-82d9d97cf980 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:49.433927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b3b3ba73-e59a-4e02-bdff-82d9d97cf980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:49.543693 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:49.621478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 659/1322] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:49.915137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3b6ba-1b69-400a-9a8e-c72afc64b0e8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:25:49.916790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-93f3b6ba-1b69-400a-9a8e-c72afc64b0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:49.938942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:50.070234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3b6ba-1b69-400a-9a8e-c72afc64b0e8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:25:50.174773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f73ae9-95d3-4a11-87a9-6c8431026364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:25:50.175375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f73ae9-95d3-4a11-87a9-6c8431026364 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:50.175712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f73ae9-95d3-4a11-87a9-6c8431026364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:50.176042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f73ae9-95d3-4a11-87a9-6c8431026364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:50.177839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-89f73ae9-95d3-4a11-87a9-6c8431026364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:50.276865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:50.333032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f73ae9-95d3-4a11-87a9-6c8431026364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:25:50.335535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f73ae9-95d3-4a11-87a9-6c8431026364 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:25:50.359978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 664/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:25:52.909695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dabc16-1d89-48ba-a1d7-f3d616cf74f5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:52.910890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dabc16-1d89-48ba-a1d7-f3d616cf74f5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:52.910890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dabc16-1d89-48ba-a1d7-f3d616cf74f5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:52.911787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 660/1324] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:52.918445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f480-3ad8-4918-8c94-45876d691f2e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:52.919415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f480-3ad8-4918-8c94-45876d691f2e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:52.919415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f480-3ad8-4918-8c94-45876d691f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:53.012259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4948f480-3ad8-4918-8c94-45876d691f2e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:53.013540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4948f480-3ad8-4918-8c94-45876d691f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:53.155393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 665/1325] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:53.473669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87df26-d45c-4263-8f04-38322e6b6d9c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:53.474214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87df26-d45c-4263-8f04-38322e6b6d9c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:53.474711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87df26-d45c-4263-8f04-38322e6b6d9c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:53.478280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 661/1326] 10.222.0.49 () {54 vars in 965 bytes} [Wed Aug 24 14:25:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:25:53.486572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a40b0-0942-4e06-a6b5-40e21debc25c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:53.487024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a40b0-0942-4e06-a6b5-40e21debc25c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:53.487437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a40b0-0942-4e06-a6b5-40e21debc25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:53.529594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:53.593840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4b6a40b0-0942-4e06-a6b5-40e21debc25c None None] MFA Rules not processed for user `48364ccf19434c4c9c09b4e5ce4bfc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:53.595354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4b6a40b0-0942-4e06-a6b5-40e21debc25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:53.713540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 666/1327] 10.222.0.49 () {58 vars in 1062 bytes} [Wed Aug 24 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:25:53.728276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87df26-d45c-4263-8f04-38322e6b6d9c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:25:53.730134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fd87df26-d45c-4263-8f04-38322e6b6d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:53.877814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87df26-d45c-4263-8f04-38322e6b6d9c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:25:53.972747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ac830-9f17-42dd-bc65-b6457288ddb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:25:53.973740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ac830-9f17-42dd-bc65-b6457288ddb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:53.973901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ac830-9f17-42dd-bc65-b6457288ddb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:53.974362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ac830-9f17-42dd-bc65-b6457288ddb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:53.976924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-974ac830-9f17-42dd-bc65-b6457288ddb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:53.999300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:54.157259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ac830-9f17-42dd-bc65-b6457288ddb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:25:54.159077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ac830-9f17-42dd-bc65-b6457288ddb4 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:25:54.182385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 662/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:25:56.746868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa083638-b0d9-4d62-8780-bc7bfeae709c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:56.747689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa083638-b0d9-4d62-8780-bc7bfeae709c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:56.748004 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa083638-b0d9-4d62-8780-bc7bfeae709c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:56.749596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 667/1329] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:25:56.756508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1a026-045a-4ae2-83d4-a3a97bb37df8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:56.757352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1a026-045a-4ae2-83d4-a3a97bb37df8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:56.757352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1a026-045a-4ae2-83d4-a3a97bb37df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:56.845275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c0b1a026-045a-4ae2-83d4-a3a97bb37df8 None None] MFA Rules not processed for user `7f091d543cf04f68b41e4456077bc426`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:25:56.846589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c0b1a026-045a-4ae2-83d4-a3a97bb37df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:56.953505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:57.001809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 663/1330] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:25:57.295100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa083638-b0d9-4d62-8780-bc7bfeae709c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:25:57.296962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-aa083638-b0d9-4d62-8780-bc7bfeae709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:57.497288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa083638-b0d9-4d62-8780-bc7bfeae709c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:25:57.530274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:25:57.603146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c6f7-e182-4280-8f04-eae22d867f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:25:57.603790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c6f7-e182-4280-8f04-eae22d867f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:25:57.604051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c6f7-e182-4280-8f04-eae22d867f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:25:57.604324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c6f7-e182-4280-8f04-eae22d867f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:25:57.606190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7042c6f7-e182-4280-8f04-eae22d867f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:25:57.793323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042c6f7-e182-4280-8f04-eae22d867f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:25:57.794864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042c6f7-e182-4280-8f04-eae22d867f90 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:25:57.819592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 668/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:25:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:26:00.701114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d577f1-0652-44ad-b411-3b41409c5434 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:00.701686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d577f1-0652-44ad-b411-3b41409c5434 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:00.701686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d577f1-0652-44ad-b411-3b41409c5434 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:00.702955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 664/1332] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:26:00.710250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3dd1-7dfa-4981-af6c-27316bfc0da2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:00.710840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3dd1-7dfa-4981-af6c-27316bfc0da2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:00.711176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3dd1-7dfa-4981-af6c-27316bfc0da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:00.789603 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:00.793056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-028e3dd1-7dfa-4981-af6c-27316bfc0da2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:00.794363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-028e3dd1-7dfa-4981-af6c-27316bfc0da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:00.915166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 669/1333] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:01.202568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d577f1-0652-44ad-b411-3b41409c5434 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:01.204246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-57d577f1-0652-44ad-b411-3b41409c5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:01.359710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d577f1-0652-44ad-b411-3b41409c5434 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:01.508589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e344bea-4ac7-4d63-b232-7cf255d7f962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:01.509539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e344bea-4ac7-4d63-b232-7cf255d7f962 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:01.510070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e344bea-4ac7-4d63-b232-7cf255d7f962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:01.510560 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e344bea-4ac7-4d63-b232-7cf255d7f962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:01.513857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7e344bea-4ac7-4d63-b232-7cf255d7f962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:01.571770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:01.689676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e344bea-4ac7-4d63-b232-7cf255d7f962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:01.691245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e344bea-4ac7-4d63-b232-7cf255d7f962 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:01.720734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 665/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:26:03.840456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff80ccf-a009-4b71-934c-4763a720f882 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:03.841033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff80ccf-a009-4b71-934c-4763a720f882 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:03.841739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff80ccf-a009-4b71-934c-4763a720f882 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:03.844958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 670/1335] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:03.855094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a868c2-34bf-45b9-ab38-169b199328e3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:03.855668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a868c2-34bf-45b9-ab38-169b199328e3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:03.855759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a868c2-34bf-45b9-ab38-169b199328e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:03.940051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a4a868c2-34bf-45b9-ab38-169b199328e3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:03.941765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a4a868c2-34bf-45b9-ab38-169b199328e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:04.031631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:04.059896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 666/1336] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:04.355333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff80ccf-a009-4b71-934c-4763a720f882 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:04.357322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0ff80ccf-a009-4b71-934c-4763a720f882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:04.493902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff80ccf-a009-4b71-934c-4763a720f882 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:04.531743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:04.588934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141815a3-af1f-4cca-a2e4-ed399d3a999c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:04.589719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141815a3-af1f-4cca-a2e4-ed399d3a999c service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:04.590246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141815a3-af1f-4cca-a2e4-ed399d3a999c service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:04.590630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141815a3-af1f-4cca-a2e4-ed399d3a999c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:04.599751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-141815a3-af1f-4cca-a2e4-ed399d3a999c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:04.744150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141815a3-af1f-4cca-a2e4-ed399d3a999c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:04.745734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141815a3-af1f-4cca-a2e4-ed399d3a999c service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:04.767563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 671/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:26:07.611324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c4af4-a9fb-4dee-a5d1-af2d09791f82 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:07.612151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c4af4-a9fb-4dee-a5d1-af2d09791f82 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:07.612151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c4af4-a9fb-4dee-a5d1-af2d09791f82 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:07.613286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 667/1338] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:26:07.623062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91b4e7-ae72-4710-8587-c5b1f9135558 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:07.623686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91b4e7-ae72-4710-8587-c5b1f9135558 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:07.624094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91b4e7-ae72-4710-8587-c5b1f9135558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:07.729030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-da91b4e7-ae72-4710-8587-c5b1f9135558 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:07.730337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-da91b4e7-ae72-4710-8587-c5b1f9135558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:07.740722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:07.855816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 672/1339] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:08.148208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c4af4-a9fb-4dee-a5d1-af2d09791f82 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:08.149952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-888c4af4-a9fb-4dee-a5d1-af2d09791f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:08.279579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c4af4-a9fb-4dee-a5d1-af2d09791f82 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:08.372448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b8a3f-aa94-48d9-b38d-2c09f30e909a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:08.373664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b8a3f-aa94-48d9-b38d-2c09f30e909a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:08.374182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b8a3f-aa94-48d9-b38d-2c09f30e909a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:08.374662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b8a3f-aa94-48d9-b38d-2c09f30e909a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:08.377830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c35b8a3f-aa94-48d9-b38d-2c09f30e909a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:08.431529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:08.521667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b8a3f-aa94-48d9-b38d-2c09f30e909a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:08.523207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b8a3f-aa94-48d9-b38d-2c09f30e909a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:08.546040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 668/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:26:13.852089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73b55-d105-4551-9fdb-dd167e804741 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:13.853458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73b55-d105-4551-9fdb-dd167e804741 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:13.853458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73b55-d105-4551-9fdb-dd167e804741 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:13.854373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 673/1341] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:13.860532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93580-e57f-4a38-be64-b914a33f3e83 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:13.861310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93580-e57f-4a38-be64-b914a33f3e83 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:13.861310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93580-e57f-4a38-be64-b914a33f3e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:13.948489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-99d93580-e57f-4a38-be64-b914a33f3e83 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:13.956401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-99d93580-e57f-4a38-be64-b914a33f3e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:14.059294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:14.074471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 669/1342] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:26:14.377490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73b55-d105-4551-9fdb-dd167e804741 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:14.381443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-eab73b55-d105-4551-9fdb-dd167e804741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:14.514158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73b55-d105-4551-9fdb-dd167e804741 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:14.566686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:14.616140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d001d38-3a9a-419b-8f41-12576762f4b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:14.617085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001d38-3a9a-419b-8f41-12576762f4b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:14.617540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001d38-3a9a-419b-8f41-12576762f4b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:14.617926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d001d38-3a9a-419b-8f41-12576762f4b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:14.620991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6d001d38-3a9a-419b-8f41-12576762f4b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:14.794295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d001d38-3a9a-419b-8f41-12576762f4b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:14.795737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d001d38-3a9a-419b-8f41-12576762f4b4 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:14.818861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 674/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:26:18.404470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f57a0-eb2d-4eef-b8df-c01835b4abf9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:18.405117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f57a0-eb2d-4eef-b8df-c01835b4abf9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:18.405413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f57a0-eb2d-4eef-b8df-c01835b4abf9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:18.407944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 670/1344] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:18.419662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f03d7b-8873-42cb-b71b-2f303df3ab0e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:18.420491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f03d7b-8873-42cb-b71b-2f303df3ab0e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:18.420678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f03d7b-8873-42cb-b71b-2f303df3ab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:18.530640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a4f03d7b-8873-42cb-b71b-2f303df3ab0e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:18.532737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a4f03d7b-8873-42cb-b71b-2f303df3ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:18.559419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:18.683867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 675/1345] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:18.989286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f57a0-eb2d-4eef-b8df-c01835b4abf9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:18.992555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b33f57a0-eb2d-4eef-b8df-c01835b4abf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:19.183157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f57a0-eb2d-4eef-b8df-c01835b4abf9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:19.300074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e247d-307d-4b6e-99a8-885c2794d261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:19.301046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e247d-307d-4b6e-99a8-885c2794d261 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:19.301519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e247d-307d-4b6e-99a8-885c2794d261 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:19.302057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e247d-307d-4b6e-99a8-885c2794d261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:19.305783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-792e247d-307d-4b6e-99a8-885c2794d261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:19.374902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:19.480910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e247d-307d-4b6e-99a8-885c2794d261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:19.483790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e247d-307d-4b6e-99a8-885c2794d261 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:19.517588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 671/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:26:25.116680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab666-b0df-4a1c-90c3-3c606b22b3cd None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:25.117442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab666-b0df-4a1c-90c3-3c606b22b3cd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:25.117442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab666-b0df-4a1c-90c3-3c606b22b3cd None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:25.118679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 676/1347] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:26:25.126185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c303cbb-7731-4f93-ae5b-a9b22e5cff3d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:25.127022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c303cbb-7731-4f93-ae5b-a9b22e5cff3d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:25.127186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c303cbb-7731-4f93-ae5b-a9b22e5cff3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:25.247557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2c303cbb-7731-4f93-ae5b-a9b22e5cff3d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:25.249112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2c303cbb-7731-4f93-ae5b-a9b22e5cff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:25.463120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:25.466968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 672/1348] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:26:25.796282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ab666-b0df-4a1c-90c3-3c606b22b3cd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:25.798338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f16ab666-b0df-4a1c-90c3-3c606b22b3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:25.952173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ab666-b0df-4a1c-90c3-3c606b22b3cd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:26.040070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:26.078167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc83ec7-c582-4da4-9440-99a250a0db3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:26.082252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc83ec7-c582-4da4-9440-99a250a0db3c service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:26.082252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc83ec7-c582-4da4-9440-99a250a0db3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:26.082252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc83ec7-c582-4da4-9440-99a250a0db3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:26.082252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5bc83ec7-c582-4da4-9440-99a250a0db3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:26.237207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc83ec7-c582-4da4-9440-99a250a0db3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:26.238694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc83ec7-c582-4da4-9440-99a250a0db3c service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:26.260253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 677/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:26:32.433130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3de24-a8af-423a-80de-f6f4d9249590 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:32.433825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3de24-a8af-423a-80de-f6f4d9249590 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:32.433825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3de24-a8af-423a-80de-f6f4d9249590 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:32.435809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 673/1350] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:32.442067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f4fe-b6b0-43f7-a34e-71f747ef8c5f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:32.442792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f4fe-b6b0-43f7-a34e-71f747ef8c5f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:32.442792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f4fe-b6b0-43f7-a34e-71f747ef8c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:32.556762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ac94f4fe-b6b0-43f7-a34e-71f747ef8c5f None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:32.558132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ac94f4fe-b6b0-43f7-a34e-71f747ef8c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:32.584175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:32.741682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 678/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:26:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:26:39.344733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffe13c-252c-448c-83ed-f26eac97dee2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:39.346349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffe13c-252c-448c-83ed-f26eac97dee2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:39.347125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffe13c-252c-448c-83ed-f26eac97dee2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:39.352495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 674/1352] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:26:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:39.358076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63924e-627b-4571-8317-582ad9f53370 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:39.358870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63924e-627b-4571-8317-582ad9f53370 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:39.359021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63924e-627b-4571-8317-582ad9f53370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:39.481282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8b63924e-627b-4571-8317-582ad9f53370 None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:39.482981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8b63924e-627b-4571-8317-582ad9f53370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:39.590704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:39.640854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 679/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:26:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:41.610169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc034b07-61cb-4587-99d9-23c7b474f08b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:41.611073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc034b07-61cb-4587-99d9-23c7b474f08b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:41.611073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc034b07-61cb-4587-99d9-23c7b474f08b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:41.613447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 675/1354] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:26:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:41.620161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0870709-9b44-42f9-a378-2e74a2c5025a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:41.620957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0870709-9b44-42f9-a378-2e74a2c5025a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:41.620957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0870709-9b44-42f9-a378-2e74a2c5025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:41.722992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a0870709-9b44-42f9-a378-2e74a2c5025a None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:41.725736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a0870709-9b44-42f9-a378-2e74a2c5025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:41.926323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 680/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:45.952752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9504b2d-a07d-45d2-838f-c2395a1b4575 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:45.953627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9504b2d-a07d-45d2-838f-c2395a1b4575 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:45.954127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9504b2d-a07d-45d2-838f-c2395a1b4575 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:45.959571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 676/1356] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:45.966619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e90a-add3-4df1-beae-a734520a7c3b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:45.967647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e90a-add3-4df1-beae-a734520a7c3b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:45.967971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e90a-add3-4df1-beae-a734520a7c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:46.037362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:46.083776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-74d1e90a-add3-4df1-beae-a734520a7c3b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:46.085948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-74d1e90a-add3-4df1-beae-a734520a7c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:46.230123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 681/1357] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:46.559070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b38df-40d6-4d73-abf9-36731535a052 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:46.559731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b38df-40d6-4d73-abf9-36731535a052 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:46.560955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b38df-40d6-4d73-abf9-36731535a052 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:46.564400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 677/1358] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 24 14:26:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:46.581830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95f0f-d010-4474-819b-32a25496a37f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:46.583205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95f0f-d010-4474-819b-32a25496a37f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:46.583662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95f0f-d010-4474-819b-32a25496a37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:46.693081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a1e95f0f-d010-4474-819b-32a25496a37f None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:46.694166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a1e95f0f-d010-4474-819b-32a25496a37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:46.763995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:46.865552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 682/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 24 14:26:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:46.883901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b38df-40d6-4d73-abf9-36731535a052 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:46.887094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e18b38df-40d6-4d73-abf9-36731535a052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:47.102976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b38df-40d6-4d73-abf9-36731535a052 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:47.204291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:47.255789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde538de-0887-4c10-b942-301cb0227ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:47.256424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde538de-0887-4c10-b942-301cb0227ac7 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:47.256728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde538de-0887-4c10-b942-301cb0227ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:47.257107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde538de-0887-4c10-b942-301cb0227ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:47.258938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cde538de-0887-4c10-b942-301cb0227ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:47.386412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde538de-0887-4c10-b942-301cb0227ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:47.388285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde538de-0887-4c10-b942-301cb0227ac7 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:47.418106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 678/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:26:50.396579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ceb9f-9fef-4cd5-98c6-059225a78fba None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:50.397378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ceb9f-9fef-4cd5-98c6-059225a78fba None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:50.397806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ceb9f-9fef-4cd5-98c6-059225a78fba None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:50.401705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 683/1361] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:26:50.408855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb5665-f43f-49ec-96bd-13077be6ecf5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:50.409703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb5665-f43f-49ec-96bd-13077be6ecf5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:50.410198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb5665-f43f-49ec-96bd-13077be6ecf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:50.522403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-faeb5665-f43f-49ec-96bd-13077be6ecf5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:50.523963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-faeb5665-f43f-49ec-96bd-13077be6ecf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:50.589414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:50.670975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 679/1362] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:26:51.002847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ceb9f-9fef-4cd5-98c6-059225a78fba None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:51.005175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dd3ceb9f-9fef-4cd5-98c6-059225a78fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:51.244697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ceb9f-9fef-4cd5-98c6-059225a78fba None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:51.257484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:51.433836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d8f17-b3c2-4397-8b98-4bfd4172020b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:51.434828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d8f17-b3c2-4397-8b98-4bfd4172020b service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:51.434968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d8f17-b3c2-4397-8b98-4bfd4172020b service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:51.435289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d8f17-b3c2-4397-8b98-4bfd4172020b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:51.437394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cc2d8f17-b3c2-4397-8b98-4bfd4172020b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:51.580084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d8f17-b3c2-4397-8b98-4bfd4172020b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:51.581728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d8f17-b3c2-4397-8b98-4bfd4172020b service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:51.610028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 684/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:26:54.871800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d152b-f7e8-4c82-bde1-41445fd1d9ba None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:54.872720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d152b-f7e8-4c82-bde1-41445fd1d9ba None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:54.872720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d152b-f7e8-4c82-bde1-41445fd1d9ba None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:54.878052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 680/1364] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:54.884117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301deb5-26f6-42e4-af61-9495629814c7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:54.884698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301deb5-26f6-42e4-af61-9495629814c7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:54.885002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301deb5-26f6-42e4-af61-9495629814c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:54.916963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:55.018252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e301deb5-26f6-42e4-af61-9495629814c7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:55.020465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e301deb5-26f6-42e4-af61-9495629814c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:55.233459 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 685/1365] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:55.801794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76eded-023d-43ab-ba86-10bdbd684466 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:55.802401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76eded-023d-43ab-ba86-10bdbd684466 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:55.802800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76eded-023d-43ab-ba86-10bdbd684466 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:55.805481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 681/1366] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 24 14:26:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:26:55.815819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8e8e-8731-42c6-a08e-624ff3cea7b2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:55.817444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8e8e-8731-42c6-a08e-624ff3cea7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:55.818598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8e8e-8731-42c6-a08e-624ff3cea7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:56.007429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e8cb8e8e-8731-42c6-a08e-624ff3cea7b2 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:56.009534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e8cb8e8e-8731-42c6-a08e-624ff3cea7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:56.089231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:56.274056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 686/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 24 14:26:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:56.288394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76eded-023d-43ab-ba86-10bdbd684466 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:56.292114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4a76eded-023d-43ab-ba86-10bdbd684466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:56.624265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76eded-023d-43ab-ba86-10bdbd684466 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:56.668499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:56.781541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3d360-527b-4a58-b8fa-be1a30aa007a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:56.781820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3d360-527b-4a58-b8fa-be1a30aa007a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:56.782251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3d360-527b-4a58-b8fa-be1a30aa007a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:56.783836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 687/1368] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:26:56.789930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2617679-3a8c-4f45-9745-4d6d309f8f7c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:56.790300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2617679-3a8c-4f45-9745-4d6d309f8f7c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:56.790711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2617679-3a8c-4f45-9745-4d6d309f8f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:56.882301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b74d29-4174-471b-bde9-1e1e3d3e4a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:56.883255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b74d29-4174-471b-bde9-1e1e3d3e4a2d service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:56.883749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b74d29-4174-471b-bde9-1e1e3d3e4a2d service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:56.884247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b74d29-4174-471b-bde9-1e1e3d3e4a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:56.887637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-07b74d29-4174-471b-bde9-1e1e3d3e4a2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:56.904494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b2617679-3a8c-4f45-9745-4d6d309f8f7c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:56.906540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b2617679-3a8c-4f45-9745-4d6d309f8f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:57.037941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b74d29-4174-471b-bde9-1e1e3d3e4a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:57.041390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b74d29-4174-471b-bde9-1e1e3d3e4a2d service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:57.082960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 682/1369] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 796 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:26:57.115824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:57.128843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 688/1370] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:26:57.862872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa14d0-f739-4cac-afad-c2358c66f3a9 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:57.864004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa14d0-f739-4cac-afad-c2358c66f3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:57.864004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa14d0-f739-4cac-afad-c2358c66f3a9 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:57.866239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 683/1371] 10.222.0.49 () {54 vars in 964 bytes} [Wed Aug 24 14:26:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:26:57.871801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df10ee-f846-4d54-9201-41666ead3b51 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:57.872211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df10ee-f846-4d54-9201-41666ead3b51 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:57.872342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df10ee-f846-4d54-9201-41666ead3b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:57.970152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e2df10ee-f846-4d54-9201-41666ead3b51 None None] MFA Rules not processed for user `9daf5be8a3dc48e78294882c62b8e60f`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:26:57.971441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e2df10ee-f846-4d54-9201-41666ead3b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:58.071974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 689/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Aug 24 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:26:58.081040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa14d0-f739-4cac-afad-c2358c66f3a9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:26:58.084749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3caa14d0-f739-4cac-afad-c2358c66f3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:58.119815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:58.220677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa14d0-f739-4cac-afad-c2358c66f3a9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:26:58.358287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58524e-3fd6-46ec-8fd0-cd922f58c950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:26:58.359416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58524e-3fd6-46ec-8fd0-cd922f58c950 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:26:58.359416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58524e-3fd6-46ec-8fd0-cd922f58c950 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:26:58.359416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58524e-3fd6-46ec-8fd0-cd922f58c950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:26:58.362198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5a58524e-3fd6-46ec-8fd0-cd922f58c950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:26:58.515419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:26:58.563701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58524e-3fd6-46ec-8fd0-cd922f58c950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:26:58.565573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58524e-3fd6-46ec-8fd0-cd922f58c950 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:26:58.591197 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 684/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:00.186719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d27e80-be1b-4e6e-8a31-508d55dc458a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:00.187320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d27e80-be1b-4e6e-8a31-508d55dc458a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:00.187817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d27e80-be1b-4e6e-8a31-508d55dc458a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:00.190455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 690/1374] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:00.197210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d50-5b17-4827-a904-68fd2e8c984b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:00.198034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d50-5b17-4827-a904-68fd2e8c984b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:00.198034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d50-5b17-4827-a904-68fd2e8c984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:00.296378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c0349d50-5b17-4827-a904-68fd2e8c984b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:00.297658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c0349d50-5b17-4827-a904-68fd2e8c984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:00.433104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 685/1375] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:27:00.732824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d27e80-be1b-4e6e-8a31-508d55dc458a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:00.734404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f9d27e80-be1b-4e6e-8a31-508d55dc458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:00.857674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:00.974342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d27e80-be1b-4e6e-8a31-508d55dc458a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:01.184667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b7148-8b11-4793-a13b-20c65b4b3d1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:01.185433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7148-8b11-4793-a13b-20c65b4b3d1f service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:01.185808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7148-8b11-4793-a13b-20c65b4b3d1f service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:01.186227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7148-8b11-4793-a13b-20c65b4b3d1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:01.188262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fc8b7148-8b11-4793-a13b-20c65b4b3d1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:01.265363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:01.329524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b7148-8b11-4793-a13b-20c65b4b3d1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:01.331299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b7148-8b11-4793-a13b-20c65b4b3d1f service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:01.357180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 691/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:04.272336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd4964-bc41-4489-9b15-1568cc6e34dc None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:04.272336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd4964-bc41-4489-9b15-1568cc6e34dc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:04.272968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd4964-bc41-4489-9b15-1568cc6e34dc None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:04.274937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 686/1377] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:04.280862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb3331-b223-4e99-8921-2f5e00336d1d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:04.281767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb3331-b223-4e99-8921-2f5e00336d1d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:04.282172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb3331-b223-4e99-8921-2f5e00336d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:04.366865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9aeb3331-b223-4e99-8921-2f5e00336d1d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:04.368234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9aeb3331-b223-4e99-8921-2f5e00336d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:04.558490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 692/1378] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:04.859751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd4964-bc41-4489-9b15-1568cc6e34dc None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:04.861318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f1cd4964-bc41-4489-9b15-1568cc6e34dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:04.909244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:05.088405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd4964-bc41-4489-9b15-1568cc6e34dc None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:05.258819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:05.264036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ac030-6f84-456f-bdc4-45615f917470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:05.264656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac030-6f84-456f-bdc4-45615f917470 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:05.264922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac030-6f84-456f-bdc4-45615f917470 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:05.265195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac030-6f84-456f-bdc4-45615f917470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:05.267342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-115ac030-6f84-456f-bdc4-45615f917470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:05.412516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ac030-6f84-456f-bdc4-45615f917470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:05.414192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ac030-6f84-456f-bdc4-45615f917470 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:05.436510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 687/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:27:08.378275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ade7-fb2f-4643-bc2f-95e71afb7e83 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:08.379513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ade7-fb2f-4643-bc2f-95e71afb7e83 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:08.379634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ade7-fb2f-4643-bc2f-95e71afb7e83 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:08.381301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 693/1380] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:08.387507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c807090-cfc4-4959-82c8-9ddebc9dfe57 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:08.388408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c807090-cfc4-4959-82c8-9ddebc9dfe57 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:08.388408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c807090-cfc4-4959-82c8-9ddebc9dfe57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:08.479301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0c807090-cfc4-4959-82c8-9ddebc9dfe57 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:08.481272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0c807090-cfc4-4959-82c8-9ddebc9dfe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:08.596007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:08.657066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 688/1381] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:09.081405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ade7-fb2f-4643-bc2f-95e71afb7e83 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:09.084407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1841ade7-fb2f-4643-bc2f-95e71afb7e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:09.119866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:09.318528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ade7-fb2f-4643-bc2f-95e71afb7e83 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:09.523046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:09.546053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac844b40-b804-4a29-8bcc-0efa79c7a41d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:09.546790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac844b40-b804-4a29-8bcc-0efa79c7a41d service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:09.547059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac844b40-b804-4a29-8bcc-0efa79c7a41d service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:09.547311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac844b40-b804-4a29-8bcc-0efa79c7a41d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:09.549091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ac844b40-b804-4a29-8bcc-0efa79c7a41d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:09.689958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac844b40-b804-4a29-8bcc-0efa79c7a41d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:09.693121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac844b40-b804-4a29-8bcc-0efa79c7a41d service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:09.716920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 694/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:12.555244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3cf2-afa1-48d0-9746-756262a8e2e5 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:12.556030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3cf2-afa1-48d0-9746-756262a8e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:12.556030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3cf2-afa1-48d0-9746-756262a8e2e5 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:12.557353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 689/1383] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:12.564224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e574d-67a4-4ca4-81bd-f3d4405ab454 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:12.564904 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e574d-67a4-4ca4-81bd-f3d4405ab454 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:12.565173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e574d-67a4-4ca4-81bd-f3d4405ab454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:12.671250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-060e574d-67a4-4ca4-81bd-f3d4405ab454 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:12.673232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-060e574d-67a4-4ca4-81bd-f3d4405ab454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:12.786122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:12.854442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 695/1384] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:13.221157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3cf2-afa1-48d0-9746-756262a8e2e5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:13.223815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-608c3cf2-afa1-48d0-9746-756262a8e2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:13.468917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3cf2-afa1-48d0-9746-756262a8e2e5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:13.554204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:13.648204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380128a4-e7c3-44a6-8c31-c700fac27618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:13.649643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380128a4-e7c3-44a6-8c31-c700fac27618 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:13.650070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380128a4-e7c3-44a6-8c31-c700fac27618 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:13.651025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380128a4-e7c3-44a6-8c31-c700fac27618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:13.652201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-380128a4-e7c3-44a6-8c31-c700fac27618 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:13.789044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380128a4-e7c3-44a6-8c31-c700fac27618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:13.793784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380128a4-e7c3-44a6-8c31-c700fac27618 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:13.828157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 690/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:15.972680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31b392-b151-48b1-b1f7-6f7de95df679 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:15.973431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31b392-b151-48b1-b1f7-6f7de95df679 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:15.973759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31b392-b151-48b1-b1f7-6f7de95df679 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:15.976524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 696/1386] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:15.986253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54789b0-d3a6-487f-ae53-60a4c44df9bb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:15.986900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54789b0-d3a6-487f-ae53-60a4c44df9bb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:15.987263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54789b0-d3a6-487f-ae53-60a4c44df9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:16.058702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca17c42-dc8b-4565-99e9-7c1bd432f78d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:16.059015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca17c42-dc8b-4565-99e9-7c1bd432f78d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:16.059192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca17c42-dc8b-4565-99e9-7c1bd432f78d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:16.060582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 697/1387] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:16.067117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9928059-2f11-4ff7-84c0-8ca7a3e56c4a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:16.067433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9928059-2f11-4ff7-84c0-8ca7a3e56c4a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:16.067706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9928059-2f11-4ff7-84c0-8ca7a3e56c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:16.103970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f54789b0-d3a6-487f-ae53-60a4c44df9bb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:16.105579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f54789b0-d3a6-487f-ae53-60a4c44df9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:16.116580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:16.182384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e9928059-2f11-4ff7-84c0-8ca7a3e56c4a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:16.184561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e9928059-2f11-4ff7-84c0-8ca7a3e56c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:16.234959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 691/1388] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:16.319140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 698/1389] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:27:16.601352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b7740-02c5-4022-9a4a-2ac174e4927e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:16.601679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b7740-02c5-4022-9a4a-2ac174e4927e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:16.601896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b7740-02c5-4022-9a4a-2ac174e4927e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:16.603332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 692/1390] 10.222.0.49 () {54 vars in 967 bytes} [Wed Aug 24 14:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:16.610520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691188a0-8375-4cd1-afb4-35c962e8453c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:16.611025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691188a0-8375-4cd1-afb4-35c962e8453c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:16.611507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691188a0-8375-4cd1-afb4-35c962e8453c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:16.654133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b7740-02c5-4022-9a4a-2ac174e4927e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:16.656239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c0b7740-02c5-4022-9a4a-2ac174e4927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:16.689978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:16.733850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-691188a0-8375-4cd1-afb4-35c962e8453c None None] MFA Rules not processed for user `aec389ead5164e778a9ee73e7ba7fc50`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:16.736088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-691188a0-8375-4cd1-afb4-35c962e8453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:16.840629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:16.854986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b7740-02c5-4022-9a4a-2ac174e4927e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:16.871784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 699/1391] 10.222.0.49 () {58 vars in 1064 bytes} [Wed Aug 24 14:27:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:16.885631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691188a0-8375-4cd1-afb4-35c962e8453c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:16.899168 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691188a0-8375-4cd1-afb4-35c962e8453c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:17.006896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a8144-605d-469a-bf0c-0db51c3b9076 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:17.008015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a8144-605d-469a-bf0c-0db51c3b9076 service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:17.008314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a8144-605d-469a-bf0c-0db51c3b9076 service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:17.008696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a8144-605d-469a-bf0c-0db51c3b9076 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:17.010825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5d2a8144-605d-469a-bf0c-0db51c3b9076 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:17.024472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbf4a5-1a37-4833-a687-d7d4033cd884 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:17.025375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbf4a5-1a37-4833-a687-d7d4033cd884 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:17.025375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbf4a5-1a37-4833-a687-d7d4033cd884 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:17.025699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbf4a5-1a37-4833-a687-d7d4033cd884 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:17.027420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-90dbf4a5-1a37-4833-a687-d7d4033cd884 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:17.120228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:17.139026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbf4a5-1a37-4833-a687-d7d4033cd884 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:17.140709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbf4a5-1a37-4833-a687-d7d4033cd884 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:17.157901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:17.164144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 693/1392] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:27:17.223577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a8144-605d-469a-bf0c-0db51c3b9076 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:17.226386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a8144-605d-469a-bf0c-0db51c3b9076 service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:17.265389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 700/1393] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:27:18.319943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861bcc-49df-4e86-959d-ea90402d9c06 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:18.320502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861bcc-49df-4e86-959d-ea90402d9c06 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:18.320886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861bcc-49df-4e86-959d-ea90402d9c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:18.405519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5f861bcc-49df-4e86-959d-ea90402d9c06 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:18.407061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5f861bcc-49df-4e86-959d-ea90402d9c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:18.517875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 694/1394] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:27:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:27:18.571735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ac8d-065e-4dbb-9296-2da29e95d02a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:18.572623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ac8d-065e-4dbb-9296-2da29e95d02a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:18.572623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ac8d-065e-4dbb-9296-2da29e95d02a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:18.573712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 701/1395] 10.222.0.49 () {54 vars in 943 bytes} [Wed Aug 24 14:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:18.582245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc580ec-aa5a-49c7-9f37-54027ae0520c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:18.582807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc580ec-aa5a-49c7-9f37-54027ae0520c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:18.587004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc580ec-aa5a-49c7-9f37-54027ae0520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:18.824468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-efc580ec-aa5a-49c7-9f37-54027ae0520c None None] MFA Rules not processed for user `36c4eeb768c04233bcbfc786b3ba20b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:18.826187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-efc580ec-aa5a-49c7-9f37-54027ae0520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:18.867516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:18.991395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 695/1396] 10.222.0.49 () {58 vars in 1040 bytes} [Wed Aug 24 14:27:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:19.002631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20ac8d-065e-4dbb-9296-2da29e95d02a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:19.006091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fd20ac8d-065e-4dbb-9296-2da29e95d02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:19.243767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20ac8d-065e-4dbb-9296-2da29e95d02a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:19.299493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:19.425969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7adb40-d159-4fcc-aa8c-6a0805b42f18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:19.427419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7adb40-d159-4fcc-aa8c-6a0805b42f18 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:19.427937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7adb40-d159-4fcc-aa8c-6a0805b42f18 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:19.428494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7adb40-d159-4fcc-aa8c-6a0805b42f18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:19.431849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ef7adb40-d159-4fcc-aa8c-6a0805b42f18 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:19.556856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7adb40-d159-4fcc-aa8c-6a0805b42f18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:19.559176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7adb40-d159-4fcc-aa8c-6a0805b42f18 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:19.570465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 702/1397] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:27:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:27:19.757822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac42da-6b36-4295-b34a-c0a1e0fcd98b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:19.758974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac42da-6b36-4295-b34a-c0a1e0fcd98b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:19.758974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac42da-6b36-4295-b34a-c0a1e0fcd98b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:19.760578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 696/1398] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:19.767169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05294fb8-2350-4e7d-ac6d-2a3bf3c612ff None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:19.767837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05294fb8-2350-4e7d-ac6d-2a3bf3c612ff None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:19.767837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05294fb8-2350-4e7d-ac6d-2a3bf3c612ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:19.848384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-05294fb8-2350-4e7d-ac6d-2a3bf3c612ff None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:19.849862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-05294fb8-2350-4e7d-ac6d-2a3bf3c612ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:19.861533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:19.993801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 703/1399] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:20.297715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac42da-6b36-4295-b34a-c0a1e0fcd98b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:20.300906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-15ac42da-6b36-4295-b34a-c0a1e0fcd98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:20.627492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac42da-6b36-4295-b34a-c0a1e0fcd98b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:20.634566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:20.872632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4aedf3-556b-4a1b-9e9a-f802ca980492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:20.874065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aedf3-556b-4a1b-9e9a-f802ca980492 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:20.874665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aedf3-556b-4a1b-9e9a-f802ca980492 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:20.875251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aedf3-556b-4a1b-9e9a-f802ca980492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:20.881290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8a4aedf3-556b-4a1b-9e9a-f802ca980492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:21.084719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4aedf3-556b-4a1b-9e9a-f802ca980492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:21.088238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4aedf3-556b-4a1b-9e9a-f802ca980492 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:21.124473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 697/1400] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 828 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:27:23.802799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8962161-b59a-41d6-9d70-2f47074327b7 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:23.803709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8962161-b59a-41d6-9d70-2f47074327b7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:23.803709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8962161-b59a-41d6-9d70-2f47074327b7 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:23.806987 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 704/1401] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:23.813827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b081c0-0259-424d-85e4-dd2d4f8e9a49 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:23.814407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b081c0-0259-424d-85e4-dd2d4f8e9a49 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:23.814407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b081c0-0259-424d-85e4-dd2d4f8e9a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:23.829069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:23.915901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b1b081c0-0259-424d-85e4-dd2d4f8e9a49 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:23.917489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b1b081c0-0259-424d-85e4-dd2d4f8e9a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:24.092390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 698/1402] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:27:24.415972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8962161-b59a-41d6-9d70-2f47074327b7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:24.417788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e8962161-b59a-41d6-9d70-2f47074327b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:24.630772 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:24.640553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8962161-b59a-41d6-9d70-2f47074327b7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:24.878940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fa3ee-dee6-439d-bb5a-c7e5c82f9af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:24.880261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa3ee-dee6-439d-bb5a-c7e5c82f9af8 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:24.880915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa3ee-dee6-439d-bb5a-c7e5c82f9af8 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:24.881408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa3ee-dee6-439d-bb5a-c7e5c82f9af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:24.885061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-792fa3ee-dee6-439d-bb5a-c7e5c82f9af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:25.085831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fa3ee-dee6-439d-bb5a-c7e5c82f9af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:25.089156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fa3ee-dee6-439d-bb5a-c7e5c82f9af8 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:25.118896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:25.132995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 705/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:27.645734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089fbbc-aae2-47f7-bdb2-9d550dee83a0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:27.646634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089fbbc-aae2-47f7-bdb2-9d550dee83a0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:27.646634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089fbbc-aae2-47f7-bdb2-9d550dee83a0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:27.648084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 699/1404] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:27.654897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52fa08-4494-45ff-b570-4d39ca9e8200 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:27.656304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52fa08-4494-45ff-b570-4d39ca9e8200 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:27.656777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52fa08-4494-45ff-b570-4d39ca9e8200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:27.796909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0a52fa08-4494-45ff-b570-4d39ca9e8200 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:27.799206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0a52fa08-4494-45ff-b570-4d39ca9e8200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:27.983079 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 706/1405] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:27:28.316629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089fbbc-aae2-47f7-bdb2-9d550dee83a0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:28.319712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6089fbbc-aae2-47f7-bdb2-9d550dee83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:28.495845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:28.602348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089fbbc-aae2-47f7-bdb2-9d550dee83a0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:28.777733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf00678-f852-457b-b908-f130228b1d84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:28.778914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00678-f852-457b-b908-f130228b1d84 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:28.778914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00678-f852-457b-b908-f130228b1d84 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:28.778914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00678-f852-457b-b908-f130228b1d84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:28.780790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3bf00678-f852-457b-b908-f130228b1d84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:28.863510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:28.924449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf00678-f852-457b-b908-f130228b1d84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:28.926718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf00678-f852-457b-b908-f130228b1d84 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:28.966524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 700/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:31.506914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c7640-afc3-4c8c-ae83-9f3821d4857d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:31.507848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c7640-afc3-4c8c-ae83-9f3821d4857d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:31.507848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c7640-afc3-4c8c-ae83-9f3821d4857d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:31.515187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 707/1407] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:27:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:31.521302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96f9be-950c-4f85-9352-d21d071977c0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:31.522013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96f9be-950c-4f85-9352-d21d071977c0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:31.522439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96f9be-950c-4f85-9352-d21d071977c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:31.667209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6d96f9be-950c-4f85-9352-d21d071977c0 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:31.669221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6d96f9be-950c-4f85-9352-d21d071977c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:31.874117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 701/1408] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:32.199882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c7640-afc3-4c8c-ae83-9f3821d4857d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:32.201922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b10c7640-afc3-4c8c-ae83-9f3821d4857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:32.324617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:32.441245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c7640-afc3-4c8c-ae83-9f3821d4857d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:32.595752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee748b0-ed50-470f-93cd-e09a5a247842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:32.597465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee748b0-ed50-470f-93cd-e09a5a247842 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:32.597959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee748b0-ed50-470f-93cd-e09a5a247842 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:32.598648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee748b0-ed50-470f-93cd-e09a5a247842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:32.602105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6ee748b0-ed50-470f-93cd-e09a5a247842 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:32.677712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:32.767415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee748b0-ed50-470f-93cd-e09a5a247842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:32.769904 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee748b0-ed50-470f-93cd-e09a5a247842 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:32.804416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 708/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:41.870325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:42.797915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=28 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:47.700986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81b3e-421b-411a-a70a-00498efad046 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:47.701709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81b3e-421b-411a-a70a-00498efad046 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:47.701910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81b3e-421b-411a-a70a-00498efad046 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:47.703664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 702/1410] 10.222.0.49 () {54 vars in 918 bytes} [Wed Aug 24 14:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:47.711341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14837c-cb88-4937-9f1c-eb06a73c692d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:47.714362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14837c-cb88-4937-9f1c-eb06a73c692d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:47.714362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14837c-cb88-4937-9f1c-eb06a73c692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:47.816570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-be14837c-cb88-4937-9f1c-eb06a73c692d None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:47.817867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-be14837c-cb88-4937-9f1c-eb06a73c692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:47.941753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 709/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 24 14:27:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:47.957058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d803c-8804-4e03-a454-b6921acd245b None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:47.957405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d803c-8804-4e03-a454-b6921acd245b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:47.957729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d803c-8804-4e03-a454-b6921acd245b None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:47.959767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 703/1412] 10.222.0.49 () {54 vars in 918 bytes} [Wed Aug 24 14:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:47.965057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4410c7-ad5d-4782-9943-c070f2db5886 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:47.965339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4410c7-ad5d-4782-9943-c070f2db5886 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:47.965618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4410c7-ad5d-4782-9943-c070f2db5886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:48.046638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:48.050808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6c4410c7-ad5d-4782-9943-c070f2db5886 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:48.052243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6c4410c7-ad5d-4782-9943-c070f2db5886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:48.182881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 710/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 24 14:27:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:27:50.770988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331c2f-4d89-4200-a477-7a65fc1d11ad None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:50.771829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331c2f-4d89-4200-a477-7a65fc1d11ad None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:50.771829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331c2f-4d89-4200-a477-7a65fc1d11ad None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:50.773713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:50.781015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20bbe1-a570-4c31-9563-437ec3e1d75e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:50.781649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20bbe1-a570-4c31-9563-437ec3e1d75e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:50.781931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20bbe1-a570-4c31-9563-437ec3e1d75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:50.910259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8d20bbe1-a570-4c31-9563-437ec3e1d75e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:50.912340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8d20bbe1-a570-4c31-9563-437ec3e1d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:51.016154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:51.079149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:51.429880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22331c2f-4d89-4200-a477-7a65fc1d11ad None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:51.432046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-22331c2f-4d89-4200-a477-7a65fc1d11ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:51.618289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22331c2f-4d89-4200-a477-7a65fc1d11ad None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:51.725686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:51.799968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9a96f-2183-4eda-a84f-97ec66c270c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:51.800925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a96f-2183-4eda-a84f-97ec66c270c8 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:51.801376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a96f-2183-4eda-a84f-97ec66c270c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:51.801803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a96f-2183-4eda-a84f-97ec66c270c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:51.804976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-13c9a96f-2183-4eda-a84f-97ec66c270c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:51.971587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9a96f-2183-4eda-a84f-97ec66c270c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:51.973231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9a96f-2183-4eda-a84f-97ec66c270c8 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:51.996098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 705/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:27:52.985954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe3add-e3b1-4d75-b637-f59685634541 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:52.987410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe3add-e3b1-4d75-b637-f59685634541 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:52.987410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe3add-e3b1-4d75-b637-f59685634541 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:52.988631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:27:52.994806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec2e91-2270-46f7-9a1f-e52735f653f5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:52.995390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec2e91-2270-46f7-9a1f-e52735f653f5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:52.995390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec2e91-2270-46f7-9a1f-e52735f653f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:53.090409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-eaec2e91-2270-46f7-9a1f-e52735f653f5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:53.091677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-eaec2e91-2270-46f7-9a1f-e52735f653f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:53.126012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:53.213367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:27:53.370662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe3add-e3b1-4d75-b637-f59685634541 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:53.372246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-07fe3add-e3b1-4d75-b637-f59685634541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:53.625027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe3add-e3b1-4d75-b637-f59685634541 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:53.673792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:53.767194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8bdf8-36fe-4d11-a2b9-9aba3dd38963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:53.768479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bdf8-36fe-4d11-a2b9-9aba3dd38963 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:53.768793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bdf8-36fe-4d11-a2b9-9aba3dd38963 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:53.769118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bdf8-36fe-4d11-a2b9-9aba3dd38963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:53.771153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0cf8bdf8-36fe-4d11-a2b9-9aba3dd38963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:53.886399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8bdf8-36fe-4d11-a2b9-9aba3dd38963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:53.888123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8bdf8-36fe-4d11-a2b9-9aba3dd38963 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:53.911503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 713/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:54.869574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c78d-8a4b-4d10-a207-6786af54bc43 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:54.870251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c78d-8a4b-4d10-a207-6786af54bc43 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:54.870251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c78d-8a4b-4d10-a207-6786af54bc43 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:54.871941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:54.878227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc44fa-f25d-4184-8277-445992606d33 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:54.878824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc44fa-f25d-4184-8277-445992606d33 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:54.878824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc44fa-f25d-4184-8277-445992606d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:54.944356 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:54.989303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a8dc44fa-f25d-4184-8277-445992606d33 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:54.991639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a8dc44fa-f25d-4184-8277-445992606d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:55.177222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:55.338602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c78d-8a4b-4d10-a207-6786af54bc43 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:55.340472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cb08c78d-8a4b-4d10-a207-6786af54bc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:55.539011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c78d-8a4b-4d10-a207-6786af54bc43 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:55.554471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:55.677338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68d37e-1aa8-42c8-ace1-c5e1f246e45e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:55.677967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d37e-1aa8-42c8-ace1-c5e1f246e45e service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:55.678272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d37e-1aa8-42c8-ace1-c5e1f246e45e service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:55.678541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d37e-1aa8-42c8-ace1-c5e1f246e45e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:55.680319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4e68d37e-1aa8-42c8-ace1-c5e1f246e45e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:55.789429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68d37e-1aa8-42c8-ace1-c5e1f246e45e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:55.791012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68d37e-1aa8-42c8-ace1-c5e1f246e45e service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:55.812594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 708/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:57.011878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdbe0-34ab-4cd4-8aac-71a61994b523 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:57.012618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdbe0-34ab-4cd4-8aac-71a61994b523 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:57.012618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdbe0-34ab-4cd4-8aac-71a61994b523 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:57.013860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:57.021203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c0077-3832-40e8-aaa3-754a965f9e90 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:57.021790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c0077-3832-40e8-aaa3-754a965f9e90 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:57.021790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c0077-3832-40e8-aaa3-754a965f9e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:57.045034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:57.117986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-055c0077-3832-40e8-aaa3-754a965f9e90 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:57.119267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-055c0077-3832-40e8-aaa3-754a965f9e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:57.236181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:57.398462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cdbe0-34ab-4cd4-8aac-71a61994b523 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:57.400021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e18cdbe0-34ab-4cd4-8aac-71a61994b523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:57.579179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:57.625718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cdbe0-34ab-4cd4-8aac-71a61994b523 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:57.796147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfb989-a006-40b8-8550-933e7eff705f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:57.796768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfb989-a006-40b8-8550-933e7eff705f service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:57.797045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfb989-a006-40b8-8550-933e7eff705f service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:57.797316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfb989-a006-40b8-8550-933e7eff705f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:57.799856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c9cfb989-a006-40b8-8550-933e7eff705f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:57.894497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:57.911725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfb989-a006-40b8-8550-933e7eff705f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:57.913777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfb989-a006-40b8-8550-933e7eff705f service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:57.936275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 716/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:27:58.927183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8ae1a-0213-4c86-8384-3b987560e88e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:58.927823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8ae1a-0213-4c86-8384-3b987560e88e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:58.927823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8ae1a-0213-4c86-8384-3b987560e88e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:58.929390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:27:58.936332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967fa34-f7fa-4c94-8d5d-3d345bf61300 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:58.936988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967fa34-f7fa-4c94-8d5d-3d345bf61300 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:58.937176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967fa34-f7fa-4c94-8d5d-3d345bf61300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:59.065958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4967fa34-f7fa-4c94-8d5d-3d345bf61300 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:27:59.067288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4967fa34-f7fa-4c94-8d5d-3d345bf61300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:59.189667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:27:59.355812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8ae1a-0213-4c86-8384-3b987560e88e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:27:59.357770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d5b8ae1a-0213-4c86-8384-3b987560e88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:59.479920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:59.555187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8ae1a-0213-4c86-8384-3b987560e88e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:27:59.695122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1232bca-9f12-4bce-8442-ed009b5743ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:27:59.695738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232bca-9f12-4bce-8442-ed009b5743ce service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:27:59.696016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232bca-9f12-4bce-8442-ed009b5743ce service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:27:59.696292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232bca-9f12-4bce-8442-ed009b5743ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:27:59.697970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d1232bca-9f12-4bce-8442-ed009b5743ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:27:59.758215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:27:59.807916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1232bca-9f12-4bce-8442-ed009b5743ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:27:59.809631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1232bca-9f12-4bce-8442-ed009b5743ce service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:27:59.832217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 711/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:00.792114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30e03c-b1a2-4ef8-918c-7f6ff8b6491c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:00.792705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30e03c-b1a2-4ef8-918c-7f6ff8b6491c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:00.792705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30e03c-b1a2-4ef8-918c-7f6ff8b6491c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:00.794726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:00.800890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9fc96-b72b-417a-aae7-6a26f533be35 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:00.801621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9fc96-b72b-417a-aae7-6a26f533be35 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:00.801621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9fc96-b72b-417a-aae7-6a26f533be35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:00.891613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3bc9fc96-b72b-417a-aae7-6a26f533be35 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:00.893824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3bc9fc96-b72b-417a-aae7-6a26f533be35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:01.009342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:01.297285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30e03c-b1a2-4ef8-918c-7f6ff8b6491c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:01.299457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ae30e03c-b1a2-4ef8-918c-7f6ff8b6491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:01.359158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:01.500760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30e03c-b1a2-4ef8-918c-7f6ff8b6491c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:01.720081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fc383-f054-4273-9c43-260c200f0b7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:01.721220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fc383-f054-4273-9c43-260c200f0b7c service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:01.722717 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fc383-f054-4273-9c43-260c200f0b7c service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:01.724345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fc383-f054-4273-9c43-260c200f0b7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:01.727871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-408fc383-f054-4273-9c43-260c200f0b7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:01.745027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:01.907131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fc383-f054-4273-9c43-260c200f0b7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:01.909412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fc383-f054-4273-9c43-260c200f0b7c service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:01.932010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 719/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:02.896887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaac9d-4e7c-4591-ad96-a875a3868541 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:02.897521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaac9d-4e7c-4591-ad96-a875a3868541 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:02.897521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaac9d-4e7c-4591-ad96-a875a3868541 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:02.898851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:02.904547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d81d6-b129-4927-8214-15a2e564c35c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:02.904828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d81d6-b129-4927-8214-15a2e564c35c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:02.905103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d81d6-b129-4927-8214-15a2e564c35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:02.999424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b74d81d6-b129-4927-8214-15a2e564c35c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:03.000723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b74d81d6-b129-4927-8214-15a2e564c35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:03.085984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:03.121074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:03.288121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eaac9d-4e7c-4591-ad96-a875a3868541 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:03.289557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b3eaac9d-4e7c-4591-ad96-a875a3868541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:03.310419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:03.471822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eaac9d-4e7c-4591-ad96-a875a3868541 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:03.578473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:03.614036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625242aa-d0d9-4167-a07b-0e35add9cdb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:03.615090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625242aa-d0d9-4167-a07b-0e35add9cdb2 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:03.615543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625242aa-d0d9-4167-a07b-0e35add9cdb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:03.615995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625242aa-d0d9-4167-a07b-0e35add9cdb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:03.619103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-625242aa-d0d9-4167-a07b-0e35add9cdb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:03.734562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625242aa-d0d9-4167-a07b-0e35add9cdb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:03.736827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625242aa-d0d9-4167-a07b-0e35add9cdb2 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:03.759028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 714/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:04.705512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a184627-5566-47a2-b646-b794f60406bd None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:04.706196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a184627-5566-47a2-b646-b794f60406bd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:04.706325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a184627-5566-47a2-b646-b794f60406bd None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:04.707855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:28:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:28:04.714391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9676a-8983-4f61-8fa0-990afc3b06ab None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:04.714966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9676a-8983-4f61-8fa0-990afc3b06ab None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:04.715167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9676a-8983-4f61-8fa0-990afc3b06ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:04.806144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b2e9676a-8983-4f61-8fa0-990afc3b06ab None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:04.807489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b2e9676a-8983-4f61-8fa0-990afc3b06ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:04.861905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:04.973214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:05.131802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a184627-5566-47a2-b646-b794f60406bd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:05.133573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4a184627-5566-47a2-b646-b794f60406bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:05.333277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a184627-5566-47a2-b646-b794f60406bd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:05.403788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:05.482160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b50920-db6f-4080-bf0c-c0147acdcf23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:05.482844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50920-db6f-4080-bf0c-c0147acdcf23 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:05.483124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50920-db6f-4080-bf0c-c0147acdcf23 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:05.483404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50920-db6f-4080-bf0c-c0147acdcf23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:05.485126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-61b50920-db6f-4080-bf0c-c0147acdcf23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:05.603540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b50920-db6f-4080-bf0c-c0147acdcf23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:05.605183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b50920-db6f-4080-bf0c-c0147acdcf23 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:05.627934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 722/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:06.574213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9631f-54fc-4869-9d81-648e0f20dced None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:06.574844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9631f-54fc-4869-9d81-648e0f20dced None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:06.574844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9631f-54fc-4869-9d81-648e0f20dced None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:06.576314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:28:06.583403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15400d-6638-429f-987c-7eb79e76e095 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:06.584019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15400d-6638-429f-987c-7eb79e76e095 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:06.584019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15400d-6638-429f-987c-7eb79e76e095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:06.713100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-db15400d-6638-429f-987c-7eb79e76e095 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:06.715191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-db15400d-6638-429f-987c-7eb79e76e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:06.719361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:06.840651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:07.070856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9631f-54fc-4869-9d81-648e0f20dced None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:07.072482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2fe9631f-54fc-4869-9d81-648e0f20dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:07.257946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9631f-54fc-4869-9d81-648e0f20dced None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:07.291181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:07.399290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4def5-454a-4d95-8b72-822016d2bd1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:07.399907 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4def5-454a-4d95-8b72-822016d2bd1f service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:07.400243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4def5-454a-4d95-8b72-822016d2bd1f service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:07.400515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4def5-454a-4d95-8b72-822016d2bd1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:07.402233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c6d4def5-454a-4d95-8b72-822016d2bd1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:07.544408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4def5-454a-4d95-8b72-822016d2bd1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:07.547246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4def5-454a-4d95-8b72-822016d2bd1f service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:07.585262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 717/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:09.562398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c3133-22dd-455c-9abd-440c62d6875f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:09.563334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c3133-22dd-455c-9abd-440c62d6875f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:09.563334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c3133-22dd-455c-9abd-440c62d6875f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:09.565026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:09.571637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e590-cf56-42d0-b87c-f7db44357c3a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:09.572236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e590-cf56-42d0-b87c-f7db44357c3a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:09.572236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e590-cf56-42d0-b87c-f7db44357c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:09.620999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:09.652470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1078e590-cf56-42d0-b87c-f7db44357c3a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:09.653983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1078e590-cf56-42d0-b87c-f7db44357c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:09.789861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:10.081586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c3133-22dd-455c-9abd-440c62d6875f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:10.083380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-279c3133-22dd-455c-9abd-440c62d6875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:10.215204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c3133-22dd-455c-9abd-440c62d6875f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:10.258374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:10.316363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d51b72-5c90-491a-9dc3-129cf48eff2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:10.317009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d51b72-5c90-491a-9dc3-129cf48eff2b service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:10.317284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d51b72-5c90-491a-9dc3-129cf48eff2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:10.317553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d51b72-5c90-491a-9dc3-129cf48eff2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:10.319615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-25d51b72-5c90-491a-9dc3-129cf48eff2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:10.463172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d51b72-5c90-491a-9dc3-129cf48eff2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:10.464649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d51b72-5c90-491a-9dc3-129cf48eff2b service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:10.488530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 725/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:12.755723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef9361-e7ec-4927-9fe8-85256106e0a4 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:12.756564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef9361-e7ec-4927-9fe8-85256106e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:12.757047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef9361-e7ec-4927-9fe8-85256106e0a4 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:12.758756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:12.765561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8128d3-4f5b-48c0-9439-6e4d7c421d12 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:12.766288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8128d3-4f5b-48c0-9439-6e4d7c421d12 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:12.766288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8128d3-4f5b-48c0-9439-6e4d7c421d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:12.852851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2a8128d3-4f5b-48c0-9439-6e4d7c421d12 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:12.854204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2a8128d3-4f5b-48c0-9439-6e4d7c421d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:12.870072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:13.036771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:13.334240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef9361-e7ec-4927-9fe8-85256106e0a4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:13.336072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-92ef9361-e7ec-4927-9fe8-85256106e0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:13.464209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef9361-e7ec-4927-9fe8-85256106e0a4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:13.480238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:13.555764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151a21-3278-491e-9528-1292f289e766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:13.556461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151a21-3278-491e-9528-1292f289e766 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:13.556790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151a21-3278-491e-9528-1292f289e766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:13.557113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151a21-3278-491e-9528-1292f289e766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:13.558888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ed151a21-3278-491e-9528-1292f289e766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:13.698850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151a21-3278-491e-9528-1292f289e766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:13.700320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151a21-3278-491e-9528-1292f289e766 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:13.725838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 720/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:16.270370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc096a0-9799-4e78-8f3d-b1253f449926 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:16.271017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc096a0-9799-4e78-8f3d-b1253f449926 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:16.271017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc096a0-9799-4e78-8f3d-b1253f449926 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:16.272464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:16.278956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22e2d0-944b-47e1-8ef8-06a8f6108557 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:16.279830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22e2d0-944b-47e1-8ef8-06a8f6108557 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:16.279830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22e2d0-944b-47e1-8ef8-06a8f6108557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:16.338954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:16.361370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ee22e2d0-944b-47e1-8ef8-06a8f6108557 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:16.363885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ee22e2d0-944b-47e1-8ef8-06a8f6108557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:16.475507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:28:16.765346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc096a0-9799-4e78-8f3d-b1253f449926 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:16.767294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-efc096a0-9799-4e78-8f3d-b1253f449926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:16.925918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc096a0-9799-4e78-8f3d-b1253f449926 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:17.006478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:17.075891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e12aa-93c0-4edc-9af9-4bdfdf13b760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:17.076857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e12aa-93c0-4edc-9af9-4bdfdf13b760 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:17.077363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e12aa-93c0-4edc-9af9-4bdfdf13b760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:17.077866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e12aa-93c0-4edc-9af9-4bdfdf13b760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:17.082204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ec7e12aa-93c0-4edc-9af9-4bdfdf13b760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:17.296537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e12aa-93c0-4edc-9af9-4bdfdf13b760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:17.300015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e12aa-93c0-4edc-9af9-4bdfdf13b760 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:17.338741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 728/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:28:19.505573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a2ef6-dfbb-4081-b505-3331caba705e None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:19.506185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a2ef6-dfbb-4081-b505-3331caba705e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:19.506185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a2ef6-dfbb-4081-b505-3331caba705e None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:19.507522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:19.514420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d2d1-9524-4eec-a818-f24ed66bfdf7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:19.515240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d2d1-9524-4eec-a818-f24ed66bfdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:19.515240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d2d1-9524-4eec-a818-f24ed66bfdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:19.603216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4c86d2d1-9524-4eec-a818-f24ed66bfdf7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:19.604513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4c86d2d1-9524-4eec-a818-f24ed66bfdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:19.632924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:19.734216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:20.037554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a2ef6-dfbb-4081-b505-3331caba705e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:20.039321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-572a2ef6-dfbb-4081-b505-3331caba705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:20.171793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a2ef6-dfbb-4081-b505-3331caba705e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:20.196389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:20.268322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b093-bd5e-484f-9037-38f99bb7f41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:20.269026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b093-bd5e-484f-9037-38f99bb7f41c service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:20.269383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b093-bd5e-484f-9037-38f99bb7f41c service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:20.269710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b093-bd5e-484f-9037-38f99bb7f41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:20.271534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-39c0b093-bd5e-484f-9037-38f99bb7f41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:20.417424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b093-bd5e-484f-9037-38f99bb7f41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:20.418985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b093-bd5e-484f-9037-38f99bb7f41c service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:20.440842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 723/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:28:22.622865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98460b3d-4ab7-4389-9224-abe580381268 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:22.623419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98460b3d-4ab7-4389-9224-abe580381268 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:22.623419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98460b3d-4ab7-4389-9224-abe580381268 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:22.625000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:28:22.632996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3de3c8-fe54-46b8-98b4-36e720c77f33 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:22.633556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3de3c8-fe54-46b8-98b4-36e720c77f33 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:22.633556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3de3c8-fe54-46b8-98b4-36e720c77f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:22.710281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:22.713939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3b3de3c8-fe54-46b8-98b4-36e720c77f33 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:22.715226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3b3de3c8-fe54-46b8-98b4-36e720c77f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:22.831082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:28:23.353575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98460b3d-4ab7-4389-9224-abe580381268 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:23.355347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-98460b3d-4ab7-4389-9224-abe580381268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:23.489094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98460b3d-4ab7-4389-9224-abe580381268 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:23.586654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:23.635677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987250d-d482-4d09-9b9a-6c602efc76c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:23.636427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987250d-d482-4d09-9b9a-6c602efc76c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:23.636607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987250d-d482-4d09-9b9a-6c602efc76c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:23.636790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987250d-d482-4d09-9b9a-6c602efc76c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:23.638547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9987250d-d482-4d09-9b9a-6c602efc76c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:23.773272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987250d-d482-4d09-9b9a-6c602efc76c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:23.774758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987250d-d482-4d09-9b9a-6c602efc76c1 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:23.812491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 731/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:25.978664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9d5c0-bcf9-4ae5-9597-4f7a24450390 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:25.979269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9d5c0-bcf9-4ae5-9597-4f7a24450390 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:25.979556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9d5c0-bcf9-4ae5-9597-4f7a24450390 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:25.981934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:25.988384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377d4f-badd-4e72-bd8c-8b7e11696d6c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:25.989005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377d4f-badd-4e72-bd8c-8b7e11696d6c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:25.989005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377d4f-badd-4e72-bd8c-8b7e11696d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:26.066488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-da377d4f-badd-4e72-bd8c-8b7e11696d6c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:26.067741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-da377d4f-badd-4e72-bd8c-8b7e11696d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:26.227062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:26.315646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:26.788414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9d5c0-bcf9-4ae5-9597-4f7a24450390 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:26.789974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-68e9d5c0-bcf9-4ae5-9597-4f7a24450390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:26.916413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9d5c0-bcf9-4ae5-9597-4f7a24450390 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:26.953784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:27.024236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17935d73-900e-43ea-8ef2-bd0cb3ae3488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:27.025016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17935d73-900e-43ea-8ef2-bd0cb3ae3488 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:27.025215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17935d73-900e-43ea-8ef2-bd0cb3ae3488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:27.025542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17935d73-900e-43ea-8ef2-bd0cb3ae3488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:27.027427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-17935d73-900e-43ea-8ef2-bd0cb3ae3488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:27.205923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17935d73-900e-43ea-8ef2-bd0cb3ae3488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:27.208690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17935d73-900e-43ea-8ef2-bd0cb3ae3488 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:27.248013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 726/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:28:29.623754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8d6c6-a205-4948-9c52-552283b7f892 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:29.624635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8d6c6-a205-4948-9c52-552283b7f892 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:29.624635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8d6c6-a205-4948-9c52-552283b7f892 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:29.626105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:29.636328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885bed-d69b-4649-b441-7bf514e1b6c7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:29.636969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885bed-d69b-4649-b441-7bf514e1b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:29.637290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885bed-d69b-4649-b441-7bf514e1b6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:29.768937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bd885bed-d69b-4649-b441-7bf514e1b6c7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:29.771007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bd885bed-d69b-4649-b441-7bf514e1b6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:29.787211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:29.906835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:30.207649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8d6c6-a205-4948-9c52-552283b7f892 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:30.209292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-81f8d6c6-a205-4948-9c52-552283b7f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:30.336787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8d6c6-a205-4948-9c52-552283b7f892 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:30.433325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:30.459488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca0e69-a3e8-4fe7-80bd-0ba705f519ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:30.460196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0e69-a3e8-4fe7-80bd-0ba705f519ba service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:30.460592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0e69-a3e8-4fe7-80bd-0ba705f519ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:30.460968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0e69-a3e8-4fe7-80bd-0ba705f519ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:30.462878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ecca0e69-a3e8-4fe7-80bd-0ba705f519ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:30.597958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca0e69-a3e8-4fe7-80bd-0ba705f519ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:30.600770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca0e69-a3e8-4fe7-80bd-0ba705f519ba service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:30.631752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 734/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:32.955416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211030b-6ded-4104-8a78-a979c2568206 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:32.956273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211030b-6ded-4104-8a78-a979c2568206 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:32.956273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211030b-6ded-4104-8a78-a979c2568206 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:32.957661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:32.964477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd089-2cad-4520-9feb-dd9248d4b2a1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:32.965385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd089-2cad-4520-9feb-dd9248d4b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:32.965570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd089-2cad-4520-9feb-dd9248d4b2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:33.067454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-33bdd089-2cad-4520-9feb-dd9248d4b2a1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:33.069143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-33bdd089-2cad-4520-9feb-dd9248d4b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:33.117772 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:33.188544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:33.478837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211030b-6ded-4104-8a78-a979c2568206 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:33.480462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d211030b-6ded-4104-8a78-a979c2568206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:33.605102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211030b-6ded-4104-8a78-a979c2568206 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:33.653616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:33.697661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87229a-e0b5-4c62-8232-c52527a00ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:33.698389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87229a-e0b5-4c62-8232-c52527a00ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:33.698727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87229a-e0b5-4c62-8232-c52527a00ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:33.699039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87229a-e0b5-4c62-8232-c52527a00ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:33.700771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1e87229a-e0b5-4c62-8232-c52527a00ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:33.845355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87229a-e0b5-4c62-8232-c52527a00ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:33.847103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87229a-e0b5-4c62-8232-c52527a00ee1 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:33.874936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 729/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:36.008159 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3249-0bd9-47c5-8067-a32d3e3fac65 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:36.008916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3249-0bd9-47c5-8067-a32d3e3fac65 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:36.008916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3249-0bd9-47c5-8067-a32d3e3fac65 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:36.009963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:36.017696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1034-9236-4c89-b062-5b89f3f2039b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:36.018320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1034-9236-4c89-b062-5b89f3f2039b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:36.018604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1034-9236-4c89-b062-5b89f3f2039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:36.108861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c47d1034-9236-4c89-b062-5b89f3f2039b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:36.110994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c47d1034-9236-4c89-b062-5b89f3f2039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:36.134269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:36.254116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:28:36.675122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f3249-0bd9-47c5-8067-a32d3e3fac65 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:36.677112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-762f3249-0bd9-47c5-8067-a32d3e3fac65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:36.815073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f3249-0bd9-47c5-8067-a32d3e3fac65 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:36.902452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:36.911681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33b553-fab9-4794-b8e8-1a0753eee7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:36.912512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33b553-fab9-4794-b8e8-1a0753eee7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:36.912833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33b553-fab9-4794-b8e8-1a0753eee7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:36.913148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33b553-fab9-4794-b8e8-1a0753eee7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:36.914985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-df33b553-fab9-4794-b8e8-1a0753eee7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:37.088051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33b553-fab9-4794-b8e8-1a0753eee7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:37.090404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33b553-fab9-4794-b8e8-1a0753eee7b4 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:37.117055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 737/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:28:39.225405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf5c1-1c06-4dac-90d6-eb68cea0bd2c None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:39.226038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf5c1-1c06-4dac-90d6-eb68cea0bd2c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:39.226038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf5c1-1c06-4dac-90d6-eb68cea0bd2c None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:39.227414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:28:39.233595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f824e1-8e59-441a-a3f1-ee597b364df6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:39.234202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f824e1-8e59-441a-a3f1-ee597b364df6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:39.234202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f824e1-8e59-441a-a3f1-ee597b364df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:39.313933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-19f824e1-8e59-441a-a3f1-ee597b364df6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:28:39.315325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-19f824e1-8e59-441a-a3f1-ee597b364df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:39.377133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:39.432700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:28:39.728090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acf5c1-1c06-4dac-90d6-eb68cea0bd2c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:28:39.730120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-53acf5c1-1c06-4dac-90d6-eb68cea0bd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:39.861363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acf5c1-1c06-4dac-90d6-eb68cea0bd2c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:28:39.925547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:39.958149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df683584-4e36-4a61-983b-c1239d5c4af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:28:39.958780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df683584-4e36-4a61-983b-c1239d5c4af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:28:39.959060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df683584-4e36-4a61-983b-c1239d5c4af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:28:39.959331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df683584-4e36-4a61-983b-c1239d5c4af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:28:39.961090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-df683584-4e36-4a61-983b-c1239d5c4af0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:28:40.104675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df683584-4e36-4a61-983b-c1239d5c4af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:28:40.106211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df683584-4e36-4a61-983b-c1239d5c4af0 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:28:40.130540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 732/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:28:49.430831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=22 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:28:50.127486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=66 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:01.511046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b1df-aead-412f-83ad-f092774c3ebd None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:01.511644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b1df-aead-412f-83ad-f092774c3ebd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:01.511644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b1df-aead-412f-83ad-f092774c3ebd None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:01.513033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 739/1471] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:29:01.522771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4776e0e-b57c-4a2c-a1d4-d5aa9ae83b93 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:01.523352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4776e0e-b57c-4a2c-a1d4-d5aa9ae83b93 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:01.523352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4776e0e-b57c-4a2c-a1d4-d5aa9ae83b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:01.607059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d4776e0e-b57c-4a2c-a1d4-d5aa9ae83b93 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:01.608327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d4776e0e-b57c-4a2c-a1d4-d5aa9ae83b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:01.735622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 733/1472] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:29:02.071653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbae65b-2d2f-4d5f-961e-d41f9a268ac8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:02.071940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbae65b-2d2f-4d5f-961e-d41f9a268ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:02.072220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbae65b-2d2f-4d5f-961e-d41f9a268ac8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:02.073605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 740/1473] 10.222.0.49 () {54 vars in 967 bytes} [Wed Aug 24 14:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:29:02.081507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c66aee-eac9-4ab6-b914-ebf407e5643d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:02.082217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c66aee-eac9-4ab6-b914-ebf407e5643d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:02.082719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c66aee-eac9-4ab6-b914-ebf407e5643d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:02.218515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-08c66aee-eac9-4ab6-b914-ebf407e5643d None None] MFA Rules not processed for user `aec389ead5164e778a9ee73e7ba7fc50`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:02.219865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-08c66aee-eac9-4ab6-b914-ebf407e5643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:02.246734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:02.341028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 734/1474] 10.222.0.49 () {58 vars in 1064 bytes} [Wed Aug 24 14:29:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:29:02.349821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbae65b-2d2f-4d5f-961e-d41f9a268ac8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:29:02.351481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4cbae65b-2d2f-4d5f-961e-d41f9a268ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:02.483045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbae65b-2d2f-4d5f-961e-d41f9a268ac8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:29:02.574631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ba068-209e-40e1-a867-04c7cd039c90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:29:02.575391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ba068-209e-40e1-a867-04c7cd039c90 service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:02.575546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ba068-209e-40e1-a867-04c7cd039c90 service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:02.575823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ba068-209e-40e1-a867-04c7cd039c90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:02.577583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-019ba068-209e-40e1-a867-04c7cd039c90 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:02.643625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:02.709863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ba068-209e-40e1-a867-04c7cd039c90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:29:02.711381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ba068-209e-40e1-a867-04c7cd039c90 service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:29:02.733198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 741/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:29:04.926661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed3e41-5201-4ee9-8190-bde0dbc6a9a2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:04.927353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed3e41-5201-4ee9-8190-bde0dbc6a9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:04.927645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed3e41-5201-4ee9-8190-bde0dbc6a9a2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:04.930245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 735/1476] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:29:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:29:04.937620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb5862-9db8-4f01-bea6-6ed04f097ec6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:04.938244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb5862-9db8-4f01-bea6-6ed04f097ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:04.938498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb5862-9db8-4f01-bea6-6ed04f097ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:05.059391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9eeb5862-9db8-4f01-bea6-6ed04f097ec6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:05.062166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9eeb5862-9db8-4f01-bea6-6ed04f097ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:05.206645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 742/1477] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:29:05.485192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed3e41-5201-4ee9-8190-bde0dbc6a9a2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:29:05.486813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e2ed3e41-5201-4ee9-8190-bde0dbc6a9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:05.649153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed3e41-5201-4ee9-8190-bde0dbc6a9a2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:29:05.711437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:05.779728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073420a-7a99-4722-ae45-07a8485f1350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:29:05.780995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073420a-7a99-4722-ae45-07a8485f1350 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:05.781270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073420a-7a99-4722-ae45-07a8485f1350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:05.781535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073420a-7a99-4722-ae45-07a8485f1350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:05.782221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073420a-7a99-4722-ae45-07a8485f1350 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:29:05.784130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-b073420a-7a99-4722-ae45-07a8485f1350 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:29:05.784130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:29:05.784130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:29:05.785426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073420a-7a99-4722-ae45-07a8485f1350 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:29:05.809024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 736/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 24 14:29:05 2022] GET /identity/v3/projects => generated 2553 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:29:07.865604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165b171-d8b8-4a83-8869-f5c31091a230 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:07.882089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165b171-d8b8-4a83-8869-f5c31091a230 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:07.882089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165b171-d8b8-4a83-8869-f5c31091a230 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:07.882089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 743/1479] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:29:07.882430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a1a1d-b746-4203-824c-3b54447a4b31 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:07.882430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a1a1d-b746-4203-824c-3b54447a4b31 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:07.882430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a1a1d-b746-4203-824c-3b54447a4b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:07.955588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-bf0a1a1d-b746-4203-824c-3b54447a4b31 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:07.956855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bf0a1a1d-b746-4203-824c-3b54447a4b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:08.070903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 737/1480] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:29:08.383429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165b171-d8b8-4a83-8869-f5c31091a230 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:29:08.385163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d165b171-d8b8-4a83-8869-f5c31091a230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:08.398053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:08.578225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165b171-d8b8-4a83-8869-f5c31091a230 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:29:08.693627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:08.736702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5a59c-ef86-4e4f-a00e-5019007ae150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:29:08.737397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5a59c-ef86-4e4f-a00e-5019007ae150 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:08.737804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5a59c-ef86-4e4f-a00e-5019007ae150 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:08.738301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5a59c-ef86-4e4f-a00e-5019007ae150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:08.741408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dfc5a59c-ef86-4e4f-a00e-5019007ae150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:08.854572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5a59c-ef86-4e4f-a00e-5019007ae150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:29:08.856212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5a59c-ef86-4e4f-a00e-5019007ae150 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:29:08.878395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 744/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:29:11.151377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d491e40-07c7-4f34-9ff1-03c92d384af7 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:11.152033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d491e40-07c7-4f34-9ff1-03c92d384af7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:11.152033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d491e40-07c7-4f34-9ff1-03c92d384af7 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:11.153331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 738/1482] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:29:11.160021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e989-5ca0-404d-9d48-925ed7deaa1d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:11.160585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e989-5ca0-404d-9d48-925ed7deaa1d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:11.160585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e989-5ca0-404d-9d48-925ed7deaa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:11.239857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0f26e989-5ca0-404d-9d48-925ed7deaa1d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:11.241126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0f26e989-5ca0-404d-9d48-925ed7deaa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:11.299799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:11.365363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 745/1483] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:29:11.671859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d491e40-07c7-4f34-9ff1-03c92d384af7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:29:11.673642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8d491e40-07c7-4f34-9ff1-03c92d384af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:11.701377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:11.802590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d491e40-07c7-4f34-9ff1-03c92d384af7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:29:11.894555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b595cf2-3275-4a27-9ac6-4d3f3f78c97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:29:11.895606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b595cf2-3275-4a27-9ac6-4d3f3f78c97c service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:11.896096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b595cf2-3275-4a27-9ac6-4d3f3f78c97c service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:11.896582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b595cf2-3275-4a27-9ac6-4d3f3f78c97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:11.899841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b595cf2-3275-4a27-9ac6-4d3f3f78c97c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:12.007046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:12.051393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b595cf2-3275-4a27-9ac6-4d3f3f78c97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:29:12.053121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b595cf2-3275-4a27-9ac6-4d3f3f78c97c service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:29:12.074942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 739/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:29:14.156378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f451ddf-281f-45b6-a85b-c8c9c70a293f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:14.157219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f451ddf-281f-45b6-a85b-c8c9c70a293f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:14.157431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f451ddf-281f-45b6-a85b-c8c9c70a293f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:14.159229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 746/1485] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:29:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:29:14.165726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a3703-1611-462e-a51b-548df8685722 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:14.166413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a3703-1611-462e-a51b-548df8685722 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:14.166413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a3703-1611-462e-a51b-548df8685722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:14.249959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f04a3703-1611-462e-a51b-548df8685722 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:14.251253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f04a3703-1611-462e-a51b-548df8685722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:14.368166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 740/1486] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:29:14.664773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f451ddf-281f-45b6-a85b-c8c9c70a293f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:29:14.666362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2f451ddf-281f-45b6-a85b-c8c9c70a293f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:14.795991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f451ddf-281f-45b6-a85b-c8c9c70a293f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:29:14.886447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:14.888813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba6446-baff-4c5d-8da3-1fc54820f202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:29:14.890578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba6446-baff-4c5d-8da3-1fc54820f202 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:14.890848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba6446-baff-4c5d-8da3-1fc54820f202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:14.891175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba6446-baff-4c5d-8da3-1fc54820f202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:14.892892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ebba6446-baff-4c5d-8da3-1fc54820f202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:15.027504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba6446-baff-4c5d-8da3-1fc54820f202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:29:15.028861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba6446-baff-4c5d-8da3-1fc54820f202 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:29:15.051468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 747/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:29:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:29:17.822889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1b337-2ffb-4338-b263-9911bd7cc78f None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:17.823816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1b337-2ffb-4338-b263-9911bd7cc78f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:17.823816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1b337-2ffb-4338-b263-9911bd7cc78f None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:17.826343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 741/1488] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:29:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:29:17.836826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15c128-f0e9-4d38-a025-435ee641dcd9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:17.837870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15c128-f0e9-4d38-a025-435ee641dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:17.837870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15c128-f0e9-4d38-a025-435ee641dcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:17.929380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ff15c128-f0e9-4d38-a025-435ee641dcd9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:17.930855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ff15c128-f0e9-4d38-a025-435ee641dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:18.015228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:18.104203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 748/1489] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:29:18.490395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1b337-2ffb-4338-b263-9911bd7cc78f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:29:18.492351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f3a1b337-2ffb-4338-b263-9911bd7cc78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:18.544971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:18.644655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1b337-2ffb-4338-b263-9911bd7cc78f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:29:18.738548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2db26-d5e9-430f-a5f5-f2148fba1ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:29:18.739166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2db26-d5e9-430f-a5f5-f2148fba1ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:18.739449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2db26-d5e9-430f-a5f5-f2148fba1ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:18.739715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2db26-d5e9-430f-a5f5-f2148fba1ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:18.741407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b1f2db26-d5e9-430f-a5f5-f2148fba1ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:18.844371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:18.880334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2db26-d5e9-430f-a5f5-f2148fba1ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:29:18.882039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2db26-d5e9-430f-a5f5-f2148fba1ba9 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:29:18.906293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 742/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:29:21.467523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9f721-992e-4e9e-ab15-352e258abee8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:21.468170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9f721-992e-4e9e-ab15-352e258abee8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:21.468170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9f721-992e-4e9e-ab15-352e258abee8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:21.470133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 749/1491] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 24 14:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:29:21.475677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7716e-1238-456d-9430-305a3e59d3e1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:21.476160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7716e-1238-456d-9430-305a3e59d3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:21.476434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7716e-1238-456d-9430-305a3e59d3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:21.558663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-56f7716e-1238-456d-9430-305a3e59d3e1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:29:21.559942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-56f7716e-1238-456d-9430-305a3e59d3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:21.676849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 743/1492] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 24 14:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:29:21.978406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9f721-992e-4e9e-ab15-352e258abee8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:29:21.980619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b1c9f721-992e-4e9e-ab15-352e258abee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:22.112065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9f721-992e-4e9e-ab15-352e258abee8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:29:22.207089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc72f4e-2cf0-4f08-b9fb-20c7b1603d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:29:22.207919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc72f4e-2cf0-4f08-b9fb-20c7b1603d98 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:29:22.208306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc72f4e-2cf0-4f08-b9fb-20c7b1603d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:29:22.208675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc72f4e-2cf0-4f08-b9fb-20c7b1603d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:29:22.210597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-adc72f4e-2cf0-4f08-b9fb-20c7b1603d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:29:22.222499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:22.367423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc72f4e-2cf0-4f08-b9fb-20c7b1603d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:29:22.369554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc72f4e-2cf0-4f08-b9fb-20c7b1603d98 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:29:22.393415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 750/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:29:31.674470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=82 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:29:32.389471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=50 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:30:55.830590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9421a-4ba8-43af-87f1-6d317126103c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:55.831186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9421a-4ba8-43af-87f1-6d317126103c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:55.831282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9421a-4ba8-43af-87f1-6d317126103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:55.933764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7fa9421a-4ba8-43af-87f1-6d317126103c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:30:55.935100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7fa9421a-4ba8-43af-87f1-6d317126103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:30:56.097756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 744/1494] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:30:56.107447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06ef12-6ceb-4601-b166-99767e5ac8fc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.108053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06ef12-6ceb-4601-b166-99767e5ac8fc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.108053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06ef12-6ceb-4601-b166-99767e5ac8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.131982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4d06ef12-6ceb-4601-b166-99767e5ac8fc None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:30:56.133264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4d06ef12-6ceb-4601-b166-99767e5ac8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:30:56.143665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 751/1495] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:30:56.149185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9421a-4ba8-43af-87f1-6d317126103c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:56.161737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9421a-4ba8-43af-87f1-6d317126103c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:56.171040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:56.172001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.172459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.173145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.175017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:30:56.175666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:56.178230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:56.178230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:30:56.178230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:30:56.179852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:56.205339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 745/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:30:56.214794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06ef12-6ceb-4601-b166-99767e5ac8fc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:56.225214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06ef12-6ceb-4601-b166-99767e5ac8fc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:56.231154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:56.231800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.232076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.232353 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.232992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:56.234887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:56.234887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 14:30:56.234887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: Aug 24 14:30:56.236088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:56.367524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 752/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:30:56 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:30:56.373546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:56.389609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57203dfc-a451-41c0-8f33-e3ccdb22eaeb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:56.450340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:30:56.457680 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:56.458841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.463862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.463862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.463862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:56.463862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:56.463862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:30:56.463862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:30:56.463862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:56.663662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 746/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:30:56 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:30:56.664050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:56.674354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada30160-f05f-479d-9fe4-32ca81aa73d2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:56.679644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:56.680285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.680562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.680833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.681501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:56.683420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:56.683420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 14:30:56.683420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: Aug 24 14:30:56.684341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:56.699797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 753/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:30:56.707992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:56.720356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2ed8c-56d3-4f16-820d-1960d4e0a104 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:56.725481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e3c9f-bb93-4471-9695-97c2af496b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:56.726132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e3c9f-bb93-4471-9695-97c2af496b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.726413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e3c9f-bb93-4471-9695-97c2af496b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.726686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e3c9f-bb93-4471-9695-97c2af496b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.727348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e3c9f-bb93-4471-9695-97c2af496b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:56.728384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-e05e3c9f-bb93-4471-9695-97c2af496b9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:56.728384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:30:56.728384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:30:56.729861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e3c9f-bb93-4471-9695-97c2af496b9f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:56.745165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 747/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:30:56.750106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:56.762037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f0ef8-158c-435e-89fb-0a5890c636e7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:56.767740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b02dc-b3e1-47a4-84aa-50c50cc2503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:56.778535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b02dc-b3e1-47a4-84aa-50c50cc2503e admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.778535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b02dc-b3e1-47a4-84aa-50c50cc2503e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.778535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b02dc-b3e1-47a4-84aa-50c50cc2503e admin admin] PATH_INFO: `/v3/projects/23643dcc02d34238a63b4ce37a81e430/users/ef6b58b2046648d2a17b0b7344652998/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.787585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b02dc-b3e1-47a4-84aa-50c50cc2503e admin admin] RBAC: Authorizing `identity:create_grant(project_id=23643dcc02d34238a63b4ce37a81e430, user_id=ef6b58b2046648d2a17b0b7344652998, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:56.788706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-721b02dc-b3e1-47a4-84aa-50c50cc2503e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:56.788706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 14:30:56.788706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: Aug 24 14:30:56.790890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b02dc-b3e1-47a4-84aa-50c50cc2503e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:56.891700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 754/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:30:56 2022] PUT /identity/v3/projects/23643dcc02d34238a63b4ce37a81e430/users/ef6b58b2046648d2a17b0b7344652998/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:30:56.916767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73078-6235-4a1f-b850-521d5d719777 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:56.917751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73078-6235-4a1f-b850-521d5d719777 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:56.918180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73078-6235-4a1f-b850-521d5d719777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:56.953130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f9f73078-6235-4a1f-b850-521d5d719777 None None] MFA Rules not processed for user `ef6b58b2046648d2a17b0b7344652998`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:30:56.954457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f9f73078-6235-4a1f-b850-521d5d719777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:30:57.036251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 748/1502] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:30:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:30:57.052749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4dfd9-a4f7-41d9-93d5-b67e2230d7c8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:57.053089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4dfd9-a4f7-41d9-93d5-b67e2230d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:57.053403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4dfd9-a4f7-41d9-93d5-b67e2230d7c8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:57.055104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 755/1503] 10.222.0.49 () {54 vars in 964 bytes} [Wed Aug 24 14:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:30:57.061078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e182dc-b36e-4d51-a43f-8e3bf801b600 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:57.061378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e182dc-b36e-4d51-a43f-8e3bf801b600 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:57.061627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e182dc-b36e-4d51-a43f-8e3bf801b600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:57.137590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-11e182dc-b36e-4d51-a43f-8e3bf801b600 None None] MFA Rules not processed for user `9daf5be8a3dc48e78294882c62b8e60f`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:30:57.138903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-11e182dc-b36e-4d51-a43f-8e3bf801b600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:30:57.234274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 749/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Aug 24 14:30:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:30:57.242726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4dfd9-a4f7-41d9-93d5-b67e2230d7c8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:57.256224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4dfd9-a4f7-41d9-93d5-b67e2230d7c8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:57.260919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:57.261541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:57.261823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:57.262144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:57.272299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:57.273781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:57.289105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 756/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:30:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:30:57.345637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c26a4-e5bd-4220-a4b8-6a73da206635 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:57.345970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c26a4-e5bd-4220-a4b8-6a73da206635 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:57.346334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c26a4-e5bd-4220-a4b8-6a73da206635 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:57.347746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 750/1506] 10.222.0.49 () {52 vars in 830 bytes} [Wed Aug 24 14:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:30:57.354415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:57.474729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448e377-7494-4894-82f8-780f7fd09249 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:57.483224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:57.484355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:57.484783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:57.485269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] PATH_INFO: `/v3/users/ef6b58b2046648d2a17b0b7344652998` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:57.487399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6b58b2046648d2a17b0b7344652998)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:57.489325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:57.489325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 14:30:57.489325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: Aug 24 14:30:57.491475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:57.795397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6b58b2046648d2a17b0b7344652998'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:57.796700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6b58b2046648d2a17b0b7344652998'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:57.818884 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6b58b2046648d2a17b0b7344652998'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:57.819921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6b58b2046648d2a17b0b7344652998'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:57.929532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6b58b2046648d2a17b0b7344652998'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:57.960171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:30:57.969799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 757/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:30:57 2022] DELETE /identity/v3/users/ef6b58b2046648d2a17b0b7344652998 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:30:57.974388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:57.975083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:57.975083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:57.996271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:30:57.997651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:30:58.010310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:30:58.073300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 751/1508] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:30:58.084956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:58.087063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:30:58.193103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1f38a-4a16-4161-b33b-d6bdcc9a9536 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:58.196938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4b253-baad-4fbc-a0d5-8f4ef873b239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:58.197634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4b253-baad-4fbc-a0d5-8f4ef873b239 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:58.198120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4b253-baad-4fbc-a0d5-8f4ef873b239 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:58.198400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4b253-baad-4fbc-a0d5-8f4ef873b239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:58.208787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4b253-baad-4fbc-a0d5-8f4ef873b239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:58.210311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4b253-baad-4fbc-a0d5-8f4ef873b239 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:58.222588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 758/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:30:58.866516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:30:58.868672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:30:58.885316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd50272-8b25-4770-b1c4-36b24dd51350 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:30:58.898147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:30:58.899226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:30:58.899708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:30:58.900150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] PATH_INFO: `/v3/projects/23643dcc02d34238a63b4ce37a81e430` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:30:58.902426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23643dcc02d34238a63b4ce37a81e430)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:30:58.903791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:30:58.903791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:30:58.903791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:30:58.905596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:30:59.066159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23643dcc02d34238a63b4ce37a81e430', 'request_id': 'req-a060cb00-e7bd-4a3d-bb38-d1b49a150623'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:59.073956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23643dcc02d34238a63b4ce37a81e430', 'request_id': 'req-a060cb00-e7bd-4a3d-bb38-d1b49a150623'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:59.090143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23643dcc02d34238a63b4ce37a81e430', 'request_id': 'req-a060cb00-e7bd-4a3d-bb38-d1b49a150623'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:59.176615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] The token cache is being invalidate because project 23643dcc02d34238a63b4ce37a81e430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:30:59.176875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:59.177710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-a060cb00-e7bd-4a3d-bb38-d1b49a150623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:30:59.179708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 752/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:30:58 2022] DELETE /identity/v3/projects/23643dcc02d34238a63b4ce37a81e430 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:31:08.232902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:31:09.172861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:32:34.337167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e7db-72bd-4804-bc25-35d3297e343d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:34.337841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e7db-72bd-4804-bc25-35d3297e343d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:34.337841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e7db-72bd-4804-bc25-35d3297e343d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:34.339221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:32:34.346138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673565-770e-4764-8a51-8b712c8149a8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:34.346874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673565-770e-4764-8a51-8b712c8149a8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:34.346874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673565-770e-4764-8a51-8b712c8149a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:34.368011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-03673565-770e-4764-8a51-8b712c8149a8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:34.369307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-03673565-770e-4764-8a51-8b712c8149a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:34.435357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:32:34.593284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e7db-72bd-4804-bc25-35d3297e343d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:32:34.600637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d049e7db-72bd-4804-bc25-35d3297e343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:34.739716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e7db-72bd-4804-bc25-35d3297e343d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:32:34.747905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50c570-7c5f-4acb-b552-749ac5f1217c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:32:34.748618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50c570-7c5f-4acb-b552-749ac5f1217c service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:34.748947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50c570-7c5f-4acb-b552-749ac5f1217c service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:34.749271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50c570-7c5f-4acb-b552-749ac5f1217c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:34.760209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50c570-7c5f-4acb-b552-749ac5f1217c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:32:34.762024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50c570-7c5f-4acb-b552-749ac5f1217c service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:32:34.773671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 760/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:32:35.721576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b2f4-39f6-4e0a-90b0-e50937159353 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:35.722534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b2f4-39f6-4e0a-90b0-e50937159353 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:35.722534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b2f4-39f6-4e0a-90b0-e50937159353 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:35.724343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:32:35.731776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595ed19-47ce-4f36-a419-d69b1e5372cb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:35.732397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595ed19-47ce-4f36-a419-d69b1e5372cb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:35.732542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595ed19-47ce-4f36-a419-d69b1e5372cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:35.759479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3595ed19-47ce-4f36-a419-d69b1e5372cb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:35.763142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3595ed19-47ce-4f36-a419-d69b1e5372cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:35.776508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:32:35.955839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657b2f4-39f6-4e0a-90b0-e50937159353 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:32:35.982032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657b2f4-39f6-4e0a-90b0-e50937159353 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:32:35.990462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c93ae9-c318-4bb0-89e9-e187e6ac6f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:32:35.993235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c93ae9-c318-4bb0-89e9-e187e6ac6f07 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:35.993235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c93ae9-c318-4bb0-89e9-e187e6ac6f07 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:35.993666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c93ae9-c318-4bb0-89e9-e187e6ac6f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:36.010288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c93ae9-c318-4bb0-89e9-e187e6ac6f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:32:36.013339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c93ae9-c318-4bb0-89e9-e187e6ac6f07 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:32:36.030277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 755/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:32:36.176298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cc71d-e5dc-4aae-b8aa-7cccb09a2339 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:36.176802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cc71d-e5dc-4aae-b8aa-7cccb09a2339 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:36.177194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cc71d-e5dc-4aae-b8aa-7cccb09a2339 None None] PATH_INFO: `/v3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:36.178856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 762/1517] 10.222.0.58 () {54 vars in 925 bytes} [Wed Aug 24 14:32:36 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:32:36.185637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678843-7d54-46c3-8210-d78d0bab3a0d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:36.186117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678843-7d54-46c3-8210-d78d0bab3a0d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:36.186497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678843-7d54-46c3-8210-d78d0bab3a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:36.246422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a2678843-7d54-46c3-8210-d78d0bab3a0d None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:36.247626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a2678843-7d54-46c3-8210-d78d0bab3a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:36.332176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 756/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 24 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:32:40.487511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff6f56-843f-4ab9-9c62-40cadc4d4244 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:40.488183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff6f56-843f-4ab9-9c62-40cadc4d4244 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:40.488296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff6f56-843f-4ab9-9c62-40cadc4d4244 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:40.490834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 763/1519] 10.222.0.49 () {54 vars in 961 bytes} [Wed Aug 24 14:32:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:32:40.497660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21497e2f-405b-492d-81c8-072b688b7222 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:40.498338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21497e2f-405b-492d-81c8-072b688b7222 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:40.498338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21497e2f-405b-492d-81c8-072b688b7222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:40.521124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-21497e2f-405b-492d-81c8-072b688b7222 None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:40.522772 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-21497e2f-405b-492d-81c8-072b688b7222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:40.532594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 757/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Aug 24 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:32:40.541461 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff6f56-843f-4ab9-9c62-40cadc4d4244 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:32:40.557290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff6f56-843f-4ab9-9c62-40cadc4d4244 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:32:40.562646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b882881-1662-442d-aaad-c085d5b8ac12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:32:40.563271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b882881-1662-442d-aaad-c085d5b8ac12 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:40.563543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b882881-1662-442d-aaad-c085d5b8ac12 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:40.563809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b882881-1662-442d-aaad-c085d5b8ac12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:40.574120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b882881-1662-442d-aaad-c085d5b8ac12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:32:40.575820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b882881-1662-442d-aaad-c085d5b8ac12 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:32:40.588155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 764/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:32:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:32:47.950314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9dd21-348d-4970-b8c5-46bf9b419de0 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:47.950932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9dd21-348d-4970-b8c5-46bf9b419de0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:47.951044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9dd21-348d-4970-b8c5-46bf9b419de0 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:47.952768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:32:47.958487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad46df-461e-4d75-9f3c-6b47609eae84 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:47.959164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad46df-461e-4d75-9f3c-6b47609eae84 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:47.959164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad46df-461e-4d75-9f3c-6b47609eae84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:47.981047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a6ad46df-461e-4d75-9f3c-6b47609eae84 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:47.982412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a6ad46df-461e-4d75-9f3c-6b47609eae84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:47.991417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:32:48.153928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9dd21-348d-4970-b8c5-46bf9b419de0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:32:48.170665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9dd21-348d-4970-b8c5-46bf9b419de0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:32:48.177809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01795076-842c-4946-b8a9-b921ab66b988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:32:48.179661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795076-842c-4946-b8a9-b921ab66b988 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:48.180103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795076-842c-4946-b8a9-b921ab66b988 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:48.180470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795076-842c-4946-b8a9-b921ab66b988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:48.193738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795076-842c-4946-b8a9-b921ab66b988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:32:48.200758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795076-842c-4946-b8a9-b921ab66b988 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:32:48.210641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 759/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:32:49.347482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a4ee-6f19-4f2c-bd44-fb1549e8c0a0 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:49.348111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a4ee-6f19-4f2c-bd44-fb1549e8c0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:49.348349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a4ee-6f19-4f2c-bd44-fb1549e8c0a0 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:49.350823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 14:32:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:32:49.359500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86132a5-1fad-40f0-bc0d-e57ba196804e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:49.364712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86132a5-1fad-40f0-bc0d-e57ba196804e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:49.364712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86132a5-1fad-40f0-bc0d-e57ba196804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:49.385263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e86132a5-1fad-40f0-bc0d-e57ba196804e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:49.386559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e86132a5-1fad-40f0-bc0d-e57ba196804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:49.395413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 14:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:32:49.561018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0a4ee-6f19-4f2c-bd44-fb1549e8c0a0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:32:49.578914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0a4ee-6f19-4f2c-bd44-fb1549e8c0a0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:32:49.584888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbf7c6-a8b5-4669-ab86-7f42dc8e2e37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:32:49.587022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf7c6-a8b5-4669-ab86-7f42dc8e2e37 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:49.587022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf7c6-a8b5-4669-ab86-7f42dc8e2e37 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:49.587309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf7c6-a8b5-4669-ab86-7f42dc8e2e37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:49.599725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbf7c6-a8b5-4669-ab86-7f42dc8e2e37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:32:49.601598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbf7c6-a8b5-4669-ab86-7f42dc8e2e37 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:32:49.612889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 767/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:32:53.565242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b074b7c-1e39-406a-bc5b-b46f44c98df2 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:53.565852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b074b7c-1e39-406a-bc5b-b46f44c98df2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:53.566148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b074b7c-1e39-406a-bc5b-b46f44c98df2 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:53.568568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:32:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:32:53.574682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd7b59-880f-456f-ae4f-b789cd476a4c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:53.575223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd7b59-880f-456f-ae4f-b789cd476a4c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:53.575388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd7b59-880f-456f-ae4f-b789cd476a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:53.597974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c8dd7b59-880f-456f-ae4f-b789cd476a4c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:53.599318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c8dd7b59-880f-456f-ae4f-b789cd476a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:53.608223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:32:53.900877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b074b7c-1e39-406a-bc5b-b46f44c98df2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:32:53.903135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1b074b7c-1e39-406a-bc5b-b46f44c98df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:53.935428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:32:53.971961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b074b7c-1e39-406a-bc5b-b46f44c98df2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:32:53.976785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563c55a-c4ab-43af-bd73-021e7a2fb1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:32:53.977494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563c55a-c4ab-43af-bd73-021e7a2fb1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:53.977832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563c55a-c4ab-43af-bd73-021e7a2fb1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:53.978204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563c55a-c4ab-43af-bd73-021e7a2fb1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:53.989413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563c55a-c4ab-43af-bd73-021e7a2fb1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:32:53.991042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563c55a-c4ab-43af-bd73-021e7a2fb1c4 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:32:54.003211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 762/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:32:56.040819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da458ff-c4c4-47f3-a407-0fb6ce63561a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:56.041569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da458ff-c4c4-47f3-a407-0fb6ce63561a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:56.041569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da458ff-c4c4-47f3-a407-0fb6ce63561a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:56.043068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:32:56.049489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf117-35b7-4080-a829-69b9cc15e4d3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:56.050439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf117-35b7-4080-a829-69b9cc15e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:56.050607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf117-35b7-4080-a829-69b9cc15e4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:56.072680 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7e3bf117-35b7-4080-a829-69b9cc15e4d3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:32:56.073957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7e3bf117-35b7-4080-a829-69b9cc15e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:32:56.082540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:32:56.631808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da458ff-c4c4-47f3-a407-0fb6ce63561a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:32:56.651960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da458ff-c4c4-47f3-a407-0fb6ce63561a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:32:56.658433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d44f-d617-406e-9152-bb2f74716c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:32:56.659464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d44f-d617-406e-9152-bb2f74716c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:32:56.659949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d44f-d617-406e-9152-bb2f74716c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:32:56.660395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d44f-d617-406e-9152-bb2f74716c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:32:56.677520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d44f-d617-406e-9152-bb2f74716c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:32:56.680926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d44f-d617-406e-9152-bb2f74716c3d service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:32:56.700818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 770/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:33:00.555544 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fe153-7ffc-4d92-93b2-a127fa43bc3a None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:33:00.556153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fe153-7ffc-4d92-93b2-a127fa43bc3a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:33:00.556153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fe153-7ffc-4d92-93b2-a127fa43bc3a None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:33:00.557802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:33:00.564963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9183b-f625-44a8-885f-c6855c5871c3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:33:00.565512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9183b-f625-44a8-885f-c6855c5871c3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:33:00.565907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9183b-f625-44a8-885f-c6855c5871c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:33:00.579338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=74 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:33:00.601417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5dc9183b-f625-44a8-885f-c6855c5871c3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:33:00.603507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5dc9183b-f625-44a8-885f-c6855c5871c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:33:00.618519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:33:01.348931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fe153-7ffc-4d92-93b2-a127fa43bc3a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:33:01.351671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ba0fe153-7ffc-4d92-93b2-a127fa43bc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:33:01.440759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fe153-7ffc-4d92-93b2-a127fa43bc3a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:33:01.445021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcc9a-0fca-4d1d-a2c0-e15c3acf3a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:33:01.446577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcc9a-0fca-4d1d-a2c0-e15c3acf3a54 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:33:01.446955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcc9a-0fca-4d1d-a2c0-e15c3acf3a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:33:01.447280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcc9a-0fca-4d1d-a2c0-e15c3acf3a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:33:01.458300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bcc9a-0fca-4d1d-a2c0-e15c3acf3a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:33:01.459965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bcc9a-0fca-4d1d-a2c0-e15c3acf3a54 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:33:01.479319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 765/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:33:03.614467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a626aec-913b-4451-a6ad-76d1baa992e3 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:33:03.615441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a626aec-913b-4451-a6ad-76d1baa992e3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:33:03.615441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a626aec-913b-4451-a6ad-76d1baa992e3 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:33:03.616631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 14:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:33:03.623006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738da11b-16fb-4123-b17f-ce096abd7d85 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:33:03.623879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738da11b-16fb-4123-b17f-ce096abd7d85 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:33:03.623879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738da11b-16fb-4123-b17f-ce096abd7d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:33:03.646409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-738da11b-16fb-4123-b17f-ce096abd7d85 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:33:03.647727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-738da11b-16fb-4123-b17f-ce096abd7d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:33:03.656582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 14:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:33:04.251679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a626aec-913b-4451-a6ad-76d1baa992e3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:33:04.253469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2a626aec-913b-4451-a6ad-76d1baa992e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:33:04.268449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a626aec-913b-4451-a6ad-76d1baa992e3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:33:04.272261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a5b2-b3d9-4404-b960-bc9f599511d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:33:04.273693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a5b2-b3d9-4404-b960-bc9f599511d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:33:04.273693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a5b2-b3d9-4404-b960-bc9f599511d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:33:04.273693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a5b2-b3d9-4404-b960-bc9f599511d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:33:04.284210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a5b2-b3d9-4404-b960-bc9f599511d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:33:04.285943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a5b2-b3d9-4404-b960-bc9f599511d2 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:33:04.298381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 773/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:33:13.640997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=47 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:33:14.293073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=8 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:32.373829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9680a-6c05-4c81-8663-1daa577e5f5c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:32.375574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9680a-6c05-4c81-8663-1daa577e5f5c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:32.376362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9680a-6c05-4c81-8663-1daa577e5f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:32.432068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c5e9680a-6c05-4c81-8663-1daa577e5f5c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:32.434876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c5e9680a-6c05-4c81-8663-1daa577e5f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:32.456708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 767/1540] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:32.470913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb48cf3-12d0-4100-8670-340b03b435c4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:32.471809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb48cf3-12d0-4100-8670-340b03b435c4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:32.471809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb48cf3-12d0-4100-8670-340b03b435c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:32.512633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2fb48cf3-12d0-4100-8670-340b03b435c4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:32.519706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2fb48cf3-12d0-4100-8670-340b03b435c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:32.527814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 774/1541] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:32.534733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9680a-6c05-4c81-8663-1daa577e5f5c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:32.558875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9680a-6c05-4c81-8663-1daa577e5f5c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:32.572252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae54d8-6b4d-42d8-9bd8-3b3bf785401a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:32.573799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae54d8-6b4d-42d8-9bd8-3b3bf785401a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:32.574654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae54d8-6b4d-42d8-9bd8-3b3bf785401a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:32.575524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae54d8-6b4d-42d8-9bd8-3b3bf785401a admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:32.577028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae54d8-6b4d-42d8-9bd8-3b3bf785401a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:32.577926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae54d8-6b4d-42d8-9bd8-3b3bf785401a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:32.581409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae54d8-6b4d-42d8-9bd8-3b3bf785401a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:32.612047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 768/1542] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:32.624937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb48cf3-12d0-4100-8670-340b03b435c4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:32.645867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb48cf3-12d0-4100-8670-340b03b435c4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:32.661138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:32.662222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:32.662876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:32.663924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:32.665078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:32.669178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:32.708690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ce63-90a2-4158-b3ed-647b25181db4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:32.709674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ce63-90a2-4158-b3ed-647b25181db4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:32.711006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ce63-90a2-4158-b3ed-647b25181db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:32.766425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7764ce63-90a2-4158-b3ed-647b25181db4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:32.769126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7764ce63-90a2-4158-b3ed-647b25181db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:32.805244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 769/1543] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:32.826418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba23f7-c825-4d9f-9727-76117c50647a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:32.826418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba23f7-c825-4d9f-9727-76117c50647a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:32.826418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba23f7-c825-4d9f-9727-76117c50647a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:32.862081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1bba23f7-c825-4d9f-9727-76117c50647a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:32.864406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1bba23f7-c825-4d9f-9727-76117c50647a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:32.865971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 775/1544] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:32 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:32.870681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.054140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e00bb-11b8-4372-b3cd-ca180c97ac03 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.071590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8337542-bb1d-4752-aa74-8a0c7d195d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.072832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337542-bb1d-4752-aa74-8a0c7d195d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.073307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337542-bb1d-4752-aa74-8a0c7d195d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.073789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337542-bb1d-4752-aa74-8a0c7d195d96 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.074934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8337542-bb1d-4752-aa74-8a0c7d195d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.076824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-a8337542-bb1d-4752-aa74-8a0c7d195d96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:34:33.076824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 14:34:33.076824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: Aug 24 14:34:33.079737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8337542-bb1d-4752-aa74-8a0c7d195d96 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.092509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 770/1545] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:33.094670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae720-888c-4383-830c-09fe92fa5eee None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.095112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae720-888c-4383-830c-09fe92fa5eee None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.095546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae720-888c-4383-830c-09fe92fa5eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.160731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-632ae720-888c-4383-830c-09fe92fa5eee None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:33.162955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-632ae720-888c-4383-830c-09fe92fa5eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:33.184479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 771/1546] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:33.185965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae720-888c-4383-830c-09fe92fa5eee None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.210609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae720-888c-4383-830c-09fe92fa5eee None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.226822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.231704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.231704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.231704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.235530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:33.236208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.237493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 776/1547] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:32 2022] POST /identity/v3/users => generated 292 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:33.239243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8c1df-b066-4331-b3e6-c1881a885128 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.239409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.239605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8c1df-b066-4331-b3e6-c1881a885128 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.239864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8c1df-b066-4331-b3e6-c1881a885128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.277958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 772/1548] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:33.279356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.287353 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-aad8c1df-b066-4331-b3e6-c1881a885128 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:33.289226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-aad8c1df-b066-4331-b3e6-c1881a885128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:33.314448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 777/1549] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:33.315207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8c1df-b066-4331-b3e6-c1881a885128 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.322580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cfdab-ec8c-4e45-8212-89115f5a53f9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.334844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8c1df-b066-4331-b3e6-c1881a885128 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.352049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.353235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.353743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.354282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.355573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.357102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.357749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.362062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.362062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.362062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.362470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.366875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.400576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 773/1550] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:33.402066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.439115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2330cd-c8f1-46f5-a9c8-87fa6a871b63 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.456490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5088fd-58d3-4d54-9d57-7244fbbea914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.457859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5088fd-58d3-4d54-9d57-7244fbbea914 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.458235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5088fd-58d3-4d54-9d57-7244fbbea914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.458673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5088fd-58d3-4d54-9d57-7244fbbea914 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.460472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5088fd-58d3-4d54-9d57-7244fbbea914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:33.461026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5088fd-58d3-4d54-9d57-7244fbbea914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.464169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5088fd-58d3-4d54-9d57-7244fbbea914 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.488301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 774/1551] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:33.490402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d02dc0-f2b2-48bb-83cf-1aa25d213b61 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.490832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d02dc0-f2b2-48bb-83cf-1aa25d213b61 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.491295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d02dc0-f2b2-48bb-83cf-1aa25d213b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.501379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 778/1552] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:33.512978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.528525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-57d02dc0-f2b2-48bb-83cf-1aa25d213b61 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:33.531122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-57d02dc0-f2b2-48bb-83cf-1aa25d213b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:33.655378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484f060-02fe-4d7c-8f33-11dca3d30ed5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.664685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2e9f7-744f-4101-8030-b23008fc6319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.665719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2e9f7-744f-4101-8030-b23008fc6319 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.666254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2e9f7-744f-4101-8030-b23008fc6319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.666733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2e9f7-744f-4101-8030-b23008fc6319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.667960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2e9f7-744f-4101-8030-b23008fc6319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.671591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2e9f7-744f-4101-8030-b23008fc6319 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.700251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 775/1553] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:33.700942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 779/1554] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:33.702223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb632c7-af42-4119-b9d8-0171f22cfbb3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.702514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaf824-eca6-4617-9a6f-abe63e0f02f2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.703023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb632c7-af42-4119-b9d8-0171f22cfbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.703151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaf824-eca6-4617-9a6f-abe63e0f02f2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.703375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaf824-eca6-4617-9a6f-abe63e0f02f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.703845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb632c7-af42-4119-b9d8-0171f22cfbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.745718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a8eaf824-eca6-4617-9a6f-abe63e0f02f2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:33.746964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a8eaf824-eca6-4617-9a6f-abe63e0f02f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:33.752175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2bb632c7-af42-4119-b9d8-0171f22cfbb3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:33.753424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2bb632c7-af42-4119-b9d8-0171f22cfbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:33.758255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 780/1555] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:33.759814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eaf824-eca6-4617-9a6f-abe63e0f02f2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.763407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 776/1556] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:33.765509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb632c7-af42-4119-b9d8-0171f22cfbb3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.775147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eaf824-eca6-4617-9a6f-abe63e0f02f2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.781858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa53ff-47e7-4a00-8b75-2c46f188d7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.782870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa53ff-47e7-4a00-8b75-2c46f188d7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.783315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa53ff-47e7-4a00-8b75-2c46f188d7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.783735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa53ff-47e7-4a00-8b75-2c46f188d7e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.783915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb632c7-af42-4119-b9d8-0171f22cfbb3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.784754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa53ff-47e7-4a00-8b75-2c46f188d7e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.788459 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa53ff-47e7-4a00-8b75-2c46f188d7e6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.791678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:33.792648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.793108 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.793572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.794673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:33.798096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:33.923296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:33.934923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 781/1557] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/projects => generated 393 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:33.937223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc93afb-c91a-4ebe-86d6-91e2902dfc90 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:33.937721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc93afb-c91a-4ebe-86d6-91e2902dfc90 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:33.938249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc93afb-c91a-4ebe-86d6-91e2902dfc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:33.963888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 777/1558] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:33.964796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:33.971129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1dc93afb-c91a-4ebe-86d6-91e2902dfc90 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:33.977453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1dc93afb-c91a-4ebe-86d6-91e2902dfc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:33.977610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69fba-bd47-4443-b91c-f5edf1fd7c01 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:33.981634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:34.049002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.049763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.050161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.050455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] PATH_INFO: `/v3/projects/84e82b68d6da45799b52ce9a6ddaf998/users/34b1955a937f4053967cf82cedd87a2e/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.054154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 782/1559] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.054154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb5c5-a0b2-4aac-ba87-4002d50bcadf None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.054154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb5c5-a0b2-4aac-ba87-4002d50bcadf None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.054627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb5c5-a0b2-4aac-ba87-4002d50bcadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.072267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e82b68d6da45799b52ce9a6ddaf998, user_id=34b1955a937f4053967cf82cedd87a2e, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.073711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:34:34.073711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:34:34.073711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:34:34.075968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.079466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4b8eb5c5-a0b2-4aac-ba87-4002d50bcadf None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:34.080734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4b8eb5c5-a0b2-4aac-ba87-4002d50bcadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:34.090181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 783/1560] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.091632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248cc2b-028d-43f1-838d-2017d8db224b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.091806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248cc2b-028d-43f1-838d-2017d8db224b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.092043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248cc2b-028d-43f1-838d-2017d8db224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.114974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f248cc2b-028d-43f1-838d-2017d8db224b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:34.116781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f248cc2b-028d-43f1-838d-2017d8db224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:34.125127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 784/1561] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.126501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe7f2d-68cf-4c42-9b48-1155a14f2fe4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.126784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe7f2d-68cf-4c42-9b48-1155a14f2fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.127051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe7f2d-68cf-4c42-9b48-1155a14f2fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.148694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0fbe7f2d-68cf-4c42-9b48-1155a14f2fe4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:34.149915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0fbe7f2d-68cf-4c42-9b48-1155a14f2fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:34.158218 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 785/1562] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.159857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db3b16-c81d-4b1d-9f74-10426aaba8b7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.160140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db3b16-c81d-4b1d-9f74-10426aaba8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.160401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db3b16-c81d-4b1d-9f74-10426aaba8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.182549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-30db3b16-c81d-4b1d-9f74-10426aaba8b7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:34.183701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-30db3b16-c81d-4b1d-9f74-10426aaba8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:34.191990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 786/1563] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.195313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db3b16-c81d-4b1d-9f74-10426aaba8b7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.203585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db3b16-c81d-4b1d-9f74-10426aaba8b7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.208332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.208952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.209224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.209490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.210220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.212359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.219642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 778/1564] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:33 2022] PUT /identity/v3/projects/84e82b68d6da45799b52ce9a6ddaf998/users/34b1955a937f4053967cf82cedd87a2e/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:34.220685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.293460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ad5b5-d4fb-445a-b7be-1a280e02317f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.298496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6bee8-44ba-4da3-8577-b0ddf783f2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.299129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6bee8-44ba-4da3-8577-b0ddf783f2ba admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.299410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6bee8-44ba-4da3-8577-b0ddf783f2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.299683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6bee8-44ba-4da3-8577-b0ddf783f2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.300339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6bee8-44ba-4da3-8577-b0ddf783f2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.302166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6bee8-44ba-4da3-8577-b0ddf783f2ba admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.315151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 779/1565] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:34.316496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fcfd-4b83-4cc4-8713-11619cff6372 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.316782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fcfd-4b83-4cc4-8713-11619cff6372 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.317054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fcfd-4b83-4cc4-8713-11619cff6372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.333329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 787/1566] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:34.340796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.341044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7a08fcfd-4b83-4cc4-8713-11619cff6372 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:34.341044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7a08fcfd-4b83-4cc4-8713-11619cff6372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:34.351114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cfb97-46fb-4c7d-ab46-7df80b2297fe admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.360441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 780/1567] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.360705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.360705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.360950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625a76-3733-4704-851e-dfbdad628503 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.361036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.361120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625a76-3733-4704-851e-dfbdad628503 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.361204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.361289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625a76-3733-4704-851e-dfbdad628503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.361399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:34.361399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.361399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:34:34.361399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 14:34:34.361399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: Aug 24 14:34:34.361740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.379231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 788/1568] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:34.380167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.395476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-89625a76-3733-4704-851e-dfbdad628503 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:34.397583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-89625a76-3733-4704-851e-dfbdad628503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:34.397811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96d5d2-3dce-4a90-9815-12ce5e1a042b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.403495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d7dfb-52dd-494c-b025-db4ffdc120d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.404350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d7dfb-52dd-494c-b025-db4ffdc120d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.404796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d7dfb-52dd-494c-b025-db4ffdc120d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.405259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d7dfb-52dd-494c-b025-db4ffdc120d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.406189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d7dfb-52dd-494c-b025-db4ffdc120d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:34.406643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d7dfb-52dd-494c-b025-db4ffdc120d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.407987 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 781/1569] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.408967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89625a76-3733-4704-851e-dfbdad628503 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.409148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d7dfb-52dd-494c-b025-db4ffdc120d8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.425047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89625a76-3733-4704-851e-dfbdad628503 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.428676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 789/1570] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:34.431197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ff22-ec72-4d10-a4f6-19d9ac4f0059 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.431685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ff22-ec72-4d10-a4f6-19d9ac4f0059 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.432171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ff22-ec72-4d10-a4f6-19d9ac4f0059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.433136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.434255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.434810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.435521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.436713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:34.437227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.440384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.462629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 782/1571] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:34.464087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.465894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a375ff22-ec72-4d10-a4f6-19d9ac4f0059 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:34.467208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a375ff22-ec72-4d10-a4f6-19d9ac4f0059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:34.477604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 790/1572] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:34.478507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ff22-ec72-4d10-a4f6-19d9ac4f0059 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.482758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93895657-d5bc-431a-a9bb-596216770b76 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.489261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.490161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.490498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.490847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.492267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.494135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.496982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ff22-ec72-4d10-a4f6-19d9ac4f0059 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.502075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.503122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.503122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.503465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.503983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.505634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.508590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 783/1573] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:34.510026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.518691 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 791/1574] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:34.519826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.523271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfa198-ad1f-4b39-8c65-74b1d32e01ba admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.529582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.530455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.530893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c91cf6-7167-44a7-965f-adf870b62136 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.531086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.531376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.532296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:34.532788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.535015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.536042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.536670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.536933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.537197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.537836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.540381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.551513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 784/1575] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:34.552180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.563607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e561c4-9f78-4a52-97d5-14adc38a09fc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.571392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.572451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.572949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.573449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.574745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:34.575313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.578441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.602563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 785/1576] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:34.602563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.613275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d26e53-fab2-40de-b81e-f4bcb73eebf0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.619103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.621235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.621235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.621235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.621235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.622016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:34:34.622016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:34:34.622016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:34:34.623229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.689113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 792/1577] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:34.689908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.717265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:34.733298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 786/1578] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:34.734970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.750707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:34.763046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32e47-a1aa-4c17-8516-2b7cea7c0fe8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.821338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236566f-d59d-4ee6-a236-e41a9beb04d8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.827731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.828082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.828592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.829114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.844454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.845246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.845642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.846044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.846786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.848877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.915654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.917123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:34.927981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 787/1579] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:34.929540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.942382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1f845-7631-439b-9e37-7871e58029cc service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:34.948461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:34.949142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:34.952436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:34.952436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] PATH_INFO: `/v3/projects/5981da6161ac4717a5b433c31188da53/users/c888090dd9f246a79c6be54faf480ed9/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:34.956890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 793/1580] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:34 2022] POST /identity/v3/projects => generated 377 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:34.958778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:34.970244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5981da6161ac4717a5b433c31188da53, user_id=c888090dd9f246a79c6be54faf480ed9, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:34.973240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.076857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3611-79e0-4309-86ae-9fdc14fbc070 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.084161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.085755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.086268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.086809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.087812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.090288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.111883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 794/1581] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:35.111883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.118474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 788/1582] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:34 2022] PUT /identity/v3/projects/5981da6161ac4717a5b433c31188da53/users/c888090dd9f246a79c6be54faf480ed9/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:35.120358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.133167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847f677-358b-4434-9f30-92e388c606be admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.203493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48733173-7081-4832-9cdb-c2c22d5b0f46 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.212529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.213586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.214133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.214741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.216218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.220695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.238843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.239920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.240300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.240673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.241462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.243940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.353450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 789/1583] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/projects => generated 405 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:35.357229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 795/1584] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/projects => generated 371 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:35.357229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.359304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.364744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:35.385116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:35.451248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6578-c014-4c21-a14e-226ec2db2a35 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.461044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.461044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.461044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.461044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.461470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.463373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.466872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d91fdc-708d-4ce8-bcfd-4ee8548f508e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.479228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.480337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.480835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.481342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.482684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.487318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.636840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 790/1585] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:35.639066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.644122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 796/1586] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/users => generated 283 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:35.645728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.659487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0587f-d591-495e-8d19-722a94adb86a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.664307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65e7df-33f4-4e94-a2c6-b9a199bcd43e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.665744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.666620 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.667021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.667358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.668084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.671293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.672547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5055a71-4bb3-4391-9abe-b477081b9f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.673751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5055a71-4bb3-4391-9abe-b477081b9f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.674255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5055a71-4bb3-4391-9abe-b477081b9f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.674708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5055a71-4bb3-4391-9abe-b477081b9f76 admin admin] PATH_INFO: `/v3/projects/7af0efe201074713a9076e220de8ea61/users/3570472f8c2b42b480ec1d1220414bec/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.692851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5055a71-4bb3-4391-9abe-b477081b9f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af0efe201074713a9076e220de8ea61, user_id=3570472f8c2b42b480ec1d1220414bec, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.696570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5055a71-4bb3-4391-9abe-b477081b9f76 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.836875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 797/1587] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:35 2022] PUT /identity/v3/projects/7af0efe201074713a9076e220de8ea61/users/3570472f8c2b42b480ec1d1220414bec/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:35.838495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a574b9d-2a6e-48ac-bafd-66ea2b8bab7f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.838832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a574b9d-2a6e-48ac-bafd-66ea2b8bab7f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.839053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a574b9d-2a6e-48ac-bafd-66ea2b8bab7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.842666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 791/1588] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:35.842666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.854093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ff53a-a1e9-4e54-8fc5-c6912a819613 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.862111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1a574b9d-2a6e-48ac-bafd-66ea2b8bab7f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:35.863625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1a574b9d-2a6e-48ac-bafd-66ea2b8bab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:35.937577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.938972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.939509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.940001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.941270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.943691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:35.958464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 798/1589] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:35.959626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a574b9d-2a6e-48ac-bafd-66ea2b8bab7f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:35.972206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a574b9d-2a6e-48ac-bafd-66ea2b8bab7f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:35.979630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c0d64-860a-426b-8fe0-acb7f2517426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:35.980315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0d64-860a-426b-8fe0-acb7f2517426 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:35.980770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0d64-860a-426b-8fe0-acb7f2517426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:35.981054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0d64-860a-426b-8fe0-acb7f2517426 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:35.981723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c0d64-860a-426b-8fe0-acb7f2517426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:35.983727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c0d64-860a-426b-8fe0-acb7f2517426 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.042118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 792/1590] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:36.043168 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.062600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25f20-e1fd-4a41-9ab7-442e634ca8db admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.070628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.071287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.071591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.071865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.072585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.074308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.088104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 793/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.089080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.104208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a829c96-ba09-4dd6-94a9-21d2997c88d8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.106675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 799/1592] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:35 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:36.108877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.109428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.110133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.112996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.113339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.113613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.113881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.115070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.116850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.128519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 794/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.129660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.141470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:36.141875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4ef4c-a751-41b0-9b9b-5d0437211c44 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.142879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:36.148132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.149109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.149658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.150649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.151634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 800/1594] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:36.151816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.152460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.155171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.158455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:36.170137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 795/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.174684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.187874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea884ca-03f0-48ac-97d3-842e062c9df7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.193200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.193830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.194252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.195149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.195805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.197496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.209169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 796/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.210427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.221718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989e203-bef2-4d8f-be1e-7b0d45522ea6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.228457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.229071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.229518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.229941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.231002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.232280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:36.233658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.238731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0abc9-4183-4628-8364-9004d513b8eb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.245843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.246748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.247488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.248000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.252549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 797/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.254209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.265346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.269425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.273147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1981-1a1d-4da9-9fb0-c493f4035e2a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.280475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.281345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.281866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.282441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.283229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.285771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.286945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 801/1598] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:36.300848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.317758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 798/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.319628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.335492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444f937-cca8-4cda-8025-0fd86012cb78 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.343190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f672ab-9abd-44ea-854d-ee3875a78dda admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.349153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.350244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.350490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.350861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.351278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.351513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.351949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.352255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.352549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.355759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.362251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:36.371317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.374887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.377850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 802/1600] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.377850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.393379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 799/1601] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:36.395327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.396057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b0aad-e1cd-402c-879c-f0a3456b1294 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.404166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.405438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.406160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.407268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.408330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.410175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd47c-79aa-4a65-9d87-5d9d8ac768f2 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.411855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.420166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.421143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.421635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.422156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.423365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.427488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.438052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 803/1602] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.439186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.449636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 800/1603] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.451409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.455946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4216ccf-df5f-4708-ad69-d8c517856e49 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.462028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.463444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.464079 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.464567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] PATH_INFO: `/v3/projects/d2bc03440f454c15abb4aab9bf6e2374/users/7b946ecd87434ad6a00890887d234281/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.465461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a902ae-2ba3-42da-935f-adefcee417f8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.472472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.473410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.473841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.474304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.475199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.478042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.492911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 801/1604] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:36.495073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.500680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2bc03440f454c15abb4aab9bf6e2374, user_id=7b946ecd87434ad6a00890887d234281, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.507492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.511898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f682-a2fd-4bfe-99d1-644bb62c8858 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.519771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.519771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.519771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.520310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] PATH_INFO: `/v3/projects/b89d036f42664c96a0ae9942d8bee18a/users/73550004cc43418b8473ef2465f0e953/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.549455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b89d036f42664c96a0ae9942d8bee18a, user_id=73550004cc43418b8473ef2465f0e953, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.555110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.677545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 804/1605] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:36 2022] PUT /identity/v3/projects/d2bc03440f454c15abb4aab9bf6e2374/users/7b946ecd87434ad6a00890887d234281/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:36.678742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.761843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 802/1606] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:36 2022] PUT /identity/v3/projects/b89d036f42664c96a0ae9942d8bee18a/users/73550004cc43418b8473ef2465f0e953/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:36.771072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:36.790827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201ac18-343b-4697-ad20-9c69d5c7cbce admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.934944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b2ca4-852e-4918-8985-dc2b4ebd9614 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:36.944544 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ef3b-7149-41a7-82be-6a5ef6edb4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.966067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ef3b-7149-41a7-82be-6a5ef6edb4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.966067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ef3b-7149-41a7-82be-6a5ef6edb4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.966067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ef3b-7149-41a7-82be-6a5ef6edb4c9 admin admin] PATH_INFO: `/v3/projects/7be521d6de8a4275808a41cba50c36e1/users/efb54407effd46a1816e10b164848bf2/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.967770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:36.967770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:36.967770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:36.967770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] PATH_INFO: `/v3/projects/e1041161b38e4ee0b0b4d7d1f6b3994c/users/e84d3ad291644b84b204b0b3102a095a/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:36.977110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1041161b38e4ee0b0b4d7d1f6b3994c, user_id=e84d3ad291644b84b204b0b3102a095a, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.980443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:36.987664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ef3b-7149-41a7-82be-6a5ef6edb4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be521d6de8a4275808a41cba50c36e1, user_id=efb54407effd46a1816e10b164848bf2, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:36.996021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ef3b-7149-41a7-82be-6a5ef6edb4c9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:37.367482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 805/1607] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:36 2022] PUT /identity/v3/projects/e1041161b38e4ee0b0b4d7d1f6b3994c/users/e84d3ad291644b84b204b0b3102a095a/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:37.367482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:37.368469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 803/1608] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:36 2022] PUT /identity/v3/projects/7be521d6de8a4275808a41cba50c36e1/users/efb54407effd46a1816e10b164848bf2/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:37.371127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f528a94-767b-4be5-83f8-5c34355fa612 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:37.373291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f528a94-767b-4be5-83f8-5c34355fa612 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:37.373705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f528a94-767b-4be5-83f8-5c34355fa612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:37.410881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2f528a94-767b-4be5-83f8-5c34355fa612 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:37.414912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2f528a94-767b-4be5-83f8-5c34355fa612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:37.430073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:37.521860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f9cc-20d8-4577-8d16-eb8379f81536 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:37.554957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c6ef5-cd6e-440e-b605-5d7aea560e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:37.558049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c6ef5-cd6e-440e-b605-5d7aea560e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:37.559683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c6ef5-cd6e-440e-b605-5d7aea560e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:37.560364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c6ef5-cd6e-440e-b605-5d7aea560e06 admin admin] PATH_INFO: `/v3/projects/85352dbb41424d14bab00dfc491b93c5/users/32f6802c125e417791721c23e2e3c900/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:37.588196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 804/1609] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:37.590892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350574-ee9b-4676-b2b8-cdb9741faca3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:37.590892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350574-ee9b-4676-b2b8-cdb9741faca3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:37.591303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350574-ee9b-4676-b2b8-cdb9741faca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:37.604843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c6ef5-cd6e-440e-b605-5d7aea560e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85352dbb41424d14bab00dfc491b93c5, user_id=32f6802c125e417791721c23e2e3c900, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:37.614150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c6ef5-cd6e-440e-b605-5d7aea560e06 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:37.632756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9f350574-ee9b-4676-b2b8-cdb9741faca3 None None] MFA Rules not processed for user `73550004cc43418b8473ef2465f0e953`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:37.634338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9f350574-ee9b-4676-b2b8-cdb9741faca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:37.673515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:37.817066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 806/1610] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:37 2022] PUT /identity/v3/projects/85352dbb41424d14bab00dfc491b93c5/users/32f6802c125e417791721c23e2e3c900/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:37.847802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b793b-6b55-4eef-b364-042cc834a3d1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:37.850738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b793b-6b55-4eef-b364-042cc834a3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:37.852427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b793b-6b55-4eef-b364-042cc834a3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:37.877615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 805/1611] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:37.883474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4f349-f0df-49d7-8242-04c6bd5a3cae None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:37.884721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4f349-f0df-49d7-8242-04c6bd5a3cae None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:37.886242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4f349-f0df-49d7-8242-04c6bd5a3cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:37.948576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-22c4f349-f0df-49d7-8242-04c6bd5a3cae None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:37.951115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-22c4f349-f0df-49d7-8242-04c6bd5a3cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:37.951438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-966b793b-6b55-4eef-b364-042cc834a3d1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:37.953898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-966b793b-6b55-4eef-b364-042cc834a3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:38.094218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 806/1612] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:38.099259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4f349-f0df-49d7-8242-04c6bd5a3cae None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:38.124055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 807/1613] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:38.127171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00474532-a4dc-41fb-abec-19058753ac6c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:38.127731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00474532-a4dc-41fb-abec-19058753ac6c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:38.128288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00474532-a4dc-41fb-abec-19058753ac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:38.216834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-00474532-a4dc-41fb-abec-19058753ac6c None None] MFA Rules not processed for user `32f6802c125e417791721c23e2e3c900`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:38.218311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-00474532-a4dc-41fb-abec-19058753ac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:38.256379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4f349-f0df-49d7-8242-04c6bd5a3cae None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:38.265745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:38.267608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:38.268263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:38.268942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:38.281895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:38.283464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:38.307708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 807/1614] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:38.309279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:38.327864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c60df-41f3-49ac-b0e0-d96cde7f1fa2 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:38.336533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 808/1615] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:38.337522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00474532-a4dc-41fb-abec-19058753ac6c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:38.353095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:38.354350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:38.355150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:38.355911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:38.360462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:38.374447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00474532-a4dc-41fb-abec-19058753ac6c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:38.380597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:38.385583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:38.388463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:38.394463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:38.394463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:38.415180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:38.416724 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:38.430810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 809/1616] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:38.432163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:38.440931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45fe57-8e1a-4046-a3d0-b85de896db6f service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:38.446597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a5760-4f37-4413-9107-78a30b522645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:38.447066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a5760-4f37-4413-9107-78a30b522645 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:38.447403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a5760-4f37-4413-9107-78a30b522645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:38.447686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a5760-4f37-4413-9107-78a30b522645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:38.459040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a5760-4f37-4413-9107-78a30b522645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:38.462151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a5760-4f37-4413-9107-78a30b522645 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:38.473709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 810/1617] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:38.475121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e8fdc-35e8-4e98-9317-ac78f23646f9 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:38.475409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e8fdc-35e8-4e98-9317-ac78f23646f9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:38.476530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e8fdc-35e8-4e98-9317-ac78f23646f9 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:38.478108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 811/1618] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 24 14:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:34:38.484565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d44b53-4d24-4bf2-bc61-003792efd3c2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:38.484870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d44b53-4d24-4bf2-bc61-003792efd3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:38.485475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d44b53-4d24-4bf2-bc61-003792efd3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:38.614246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-96d44b53-4d24-4bf2-bc61-003792efd3c2 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:38.614246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-96d44b53-4d24-4bf2-bc61-003792efd3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:38.628718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 808/1619] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:38 2022] POST /identity/v3/projects => generated 363 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:38.628718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:39.089856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 812/1620] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 24 14:34:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:39.089856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d44b53-4d24-4bf2-bc61-003792efd3c2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:39.089856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d44b53-4d24-4bf2-bc61-003792efd3c2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:39.093008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a089c-dd0f-478d-a58c-c5a8b15b15b0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:39.093008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:39.093008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:39.093008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:39.094725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:39.094725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:39.094725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:39.095507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:39.095507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:39.095507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:39.095507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:39.096440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:39.119471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:39.123038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:39.140677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 813/1621] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:39.269439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 809/1622] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:38 2022] POST /identity/v3/users => generated 288 bytes in 645 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:39.274580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:39.297866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd640c-97b7-4339-93c1-224743e243e6 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:39.310800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f915b-28d2-4948-b098-58125988c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:39.311655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f915b-28d2-4948-b098-58125988c70e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:39.312423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f915b-28d2-4948-b098-58125988c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:39.312423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f915b-28d2-4948-b098-58125988c70e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:39.315671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f915b-28d2-4948-b098-58125988c70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:39.315671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f915b-28d2-4948-b098-58125988c70e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:39.346206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 814/1623] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:39.351616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:39.406801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa3b7e-4079-4ea7-806a-c3b943807095 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:39.421609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:39.423883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:39.425566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:39.426298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] PATH_INFO: `/v3/projects/b624c98fbe764c4ca3374feab58a5900/users/2d43abf126d14efd8f2b6dfb8148b257/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:39.484774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e12f40-8d21-4343-8a5e-67c8b3ab9a17 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:39.485070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e12f40-8d21-4343-8a5e-67c8b3ab9a17 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:39.485358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e12f40-8d21-4343-8a5e-67c8b3ab9a17 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:39.486825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 815/1624] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 24 14:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:34:39.496223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b624c98fbe764c4ca3374feab58a5900, user_id=2d43abf126d14efd8f2b6dfb8148b257, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:39.501686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14f79e-e867-43b9-a065-7ff1ee2a91ff None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:39.502172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14f79e-e867-43b9-a065-7ff1ee2a91ff None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:39.502354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14f79e-e867-43b9-a065-7ff1ee2a91ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:39.511918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:39.533718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fa14f79e-e867-43b9-a065-7ff1ee2a91ff None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:39.535476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fa14f79e-e867-43b9-a065-7ff1ee2a91ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:39.556665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 816/1625] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:34:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:39.568649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14f79e-e867-43b9-a065-7ff1ee2a91ff None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:39.710068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 810/1626] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:39 2022] PUT /identity/v3/projects/b624c98fbe764c4ca3374feab58a5900/users/2d43abf126d14efd8f2b6dfb8148b257/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:34:39.717725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:39.735522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14f79e-e867-43b9-a065-7ff1ee2a91ff None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:39.893132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa041-8225-40af-94ae-8b680702b139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:39.893812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa041-8225-40af-94ae-8b680702b139 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:39.894136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa041-8225-40af-94ae-8b680702b139 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:39.894476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa041-8225-40af-94ae-8b680702b139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:40.014570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418d8cd-1f31-4ccd-8b68-3ec327078c55 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:40.028261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:40.028749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:40.029031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:40.029339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:40.030061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:40.043065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:40.059000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 811/1627] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:40.063032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:40.087977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50b820-2d3d-47f8-9308-72a690faa695 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:40.090124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa041-8225-40af-94ae-8b680702b139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:40.100272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa041-8225-40af-94ae-8b680702b139 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:40.100660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:40.126889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:40.131256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:40.132648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:40.133143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] PATH_INFO: `/v3/projects/b624c98fbe764c4ca3374feab58a5900/users/2d43abf126d14efd8f2b6dfb8148b257/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:40.146521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b624c98fbe764c4ca3374feab58a5900, user_id=2d43abf126d14efd8f2b6dfb8148b257, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:40.157080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:40.179657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 817/1628] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:34:40.197111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 812/1629] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:40 2022] PUT /identity/v3/projects/b624c98fbe764c4ca3374feab58a5900/users/2d43abf126d14efd8f2b6dfb8148b257/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:34:40.208226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98fd18-981c-46a2-bce7-84a07810dd99 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:40.208575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98fd18-981c-46a2-bce7-84a07810dd99 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:40.208831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98fd18-981c-46a2-bce7-84a07810dd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:40.271815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6c98fd18-981c-46a2-bce7-84a07810dd99 None None] MFA Rules not processed for user `2d43abf126d14efd8f2b6dfb8148b257`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:40.274583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6c98fd18-981c-46a2-bce7-84a07810dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:40.512546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 818/1630] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:40.543425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:40.836034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17c81c-3559-4cdf-8c58-11046f770379 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:40.845394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64e754-3277-4f00-8a60-01a0d434bc6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:40.846171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64e754-3277-4f00-8a60-01a0d434bc6e service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:40.846460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64e754-3277-4f00-8a60-01a0d434bc6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:40.846799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64e754-3277-4f00-8a60-01a0d434bc6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:40.874369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64e754-3277-4f00-8a60-01a0d434bc6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:40.877346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64e754-3277-4f00-8a60-01a0d434bc6e service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:40.921229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 813/1631] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:42.901941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aa831-5a9d-4840-9091-bff98e6f05f2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:42.903597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aa831-5a9d-4840-9091-bff98e6f05f2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:42.904378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aa831-5a9d-4840-9091-bff98e6f05f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:42.958290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e30aa831-5a9d-4840-9091-bff98e6f05f2 None None] MFA Rules not processed for user `efb54407effd46a1816e10b164848bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:42.959727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e30aa831-5a9d-4840-9091-bff98e6f05f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:43.024516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292f276-fa64-45ce-9a03-bc1f69d0ef30 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.025335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292f276-fa64-45ce-9a03-bc1f69d0ef30 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.025335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292f276-fa64-45ce-9a03-bc1f69d0ef30 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.028019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 814/1632] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 24 14:34:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:34:43.036690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3f757-e95b-4944-840f-ac96fd9d96d1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.038077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3f757-e95b-4944-840f-ac96fd9d96d1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.039175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3f757-e95b-4944-840f-ac96fd9d96d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.090096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e6b3f757-e95b-4944-840f-ac96fd9d96d1 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:43.092408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e6b3f757-e95b-4944-840f-ac96fd9d96d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:43.151261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 819/1633] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:43.155947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30aa831-5a9d-4840-9091-bff98e6f05f2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:43.285753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 815/1634] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 24 14:34:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:43.294507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c922a2-5c79-4219-9422-21bd0bfe63fa None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.294794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c922a2-5c79-4219-9422-21bd0bfe63fa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.295075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c922a2-5c79-4219-9422-21bd0bfe63fa None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.306190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 816/1635] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 24 14:34:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:34:43.309188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743113b3-77d3-42e2-ac31-22f2be2f2ec2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.309525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743113b3-77d3-42e2-ac31-22f2be2f2ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.312356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743113b3-77d3-42e2-ac31-22f2be2f2ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.379572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-743113b3-77d3-42e2-ac31-22f2be2f2ec2 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:43.382310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-743113b3-77d3-42e2-ac31-22f2be2f2ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:43.388584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30aa831-5a9d-4840-9091-bff98e6f05f2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:43.403491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:43.404923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.405508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.406129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.407347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:43.412881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:43.416065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 817/1636] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:34:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:43.440961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743113b3-77d3-42e2-ac31-22f2be2f2ec2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:43.453426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743113b3-77d3-42e2-ac31-22f2be2f2ec2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:43.472151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:43.472796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.473075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.473344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.501073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:43.505353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:43.524645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:43.576948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 818/1637] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:43.582775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:43.661408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92785f04-f38a-4852-9ff2-1a7f35e54714 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:43.668053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:43.668715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.668993 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.669394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] PATH_INFO: `/v3/users/73550004cc43418b8473ef2465f0e953` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.670804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73550004cc43418b8473ef2465f0e953)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:43.671880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: WARNING py.warnings [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:34:43.671880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: warnings.warn(msg) Aug 24 14:34:43.671880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: Aug 24 14:34:43.673061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:43.695283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 820/1638] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:43 2022] POST /identity/v3/projects => generated 405 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:43.701448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:43.978969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510be287-a28b-4ceb-8436-d7abb98443d1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:43.987469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:43.988368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:43.988828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:43.989886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:43.990747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:43.995154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:44.260288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 821/1639] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:43 2022] POST /identity/v3/users => generated 309 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:44.267534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:44.309313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73550004cc43418b8473ef2465f0e953'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:44.311643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73550004cc43418b8473ef2465f0e953'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:44.313085 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73550004cc43418b8473ef2465f0e953'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:44.313604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39323302-23b6-45e9-823b-32613e0c6fab admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:44.328482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:44.329200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:44.329551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:44.329939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:44.331143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:44.334241 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:44.355040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 822/1640] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:44.365109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:44.379325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:44.386311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:44.405169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5e1e2-efa2-475f-a256-7753fe1e5155 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:44.414934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:44.416475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:44.417258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:44.418091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] PATH_INFO: `/v3/projects/51421852ecf449dbab34d7cfa974abae/users/c86ef757a41e4b9abb4fef7ef3c219b6/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:44.454697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51421852ecf449dbab34d7cfa974abae, user_id=c86ef757a41e4b9abb4fef7ef3c219b6, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:44.460195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:44.464850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73550004cc43418b8473ef2465f0e953'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:44.527494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-178a3894-59c1-4a41-9e31-765a61d6ab8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73550004cc43418b8473ef2465f0e953'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:44.615636 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 823/1641] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:44 2022] PUT /identity/v3/projects/51421852ecf449dbab34d7cfa974abae/users/c86ef757a41e4b9abb4fef7ef3c219b6/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:44.619959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:44.644365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 819/1642] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:34:43 2022] DELETE /identity/v3/users/73550004cc43418b8473ef2465f0e953 => generated 0 bytes in 1066 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:44.654360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:44.659245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:44.661895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:44.730717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:44.732101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:44.860214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318216ec-119d-4331-9e72-647278201d6c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:44.935408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 820/1643] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:44.993818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:44.997098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:45.084489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:45.093579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:45.094518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:45.096937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:45.103855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:45.106953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:45.112105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:45.145798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 824/1644] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:34:45.156166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:45.191578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d02a-1816-4ffe-80f5-022c8db05bf0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:45.219311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:45.220805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:45.221622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:45.224984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] PATH_INFO: `/v3/projects/51421852ecf449dbab34d7cfa974abae/users/c86ef757a41e4b9abb4fef7ef3c219b6/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:45.234116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51421852ecf449dbab34d7cfa974abae, user_id=c86ef757a41e4b9abb4fef7ef3c219b6, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:45.251066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:45.278182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a66ba9-2524-40ba-9010-e7f2ce44dc65 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:45.278780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 825/1645] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:45 2022] PUT /identity/v3/projects/51421852ecf449dbab34d7cfa974abae/users/c86ef757a41e4b9abb4fef7ef3c219b6/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:45.463438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a18b95-5492-406f-8c6a-a37eb87c1fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:45.464756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18b95-5492-406f-8c6a-a37eb87c1fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:45.465202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18b95-5492-406f-8c6a-a37eb87c1fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:45.470135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18b95-5492-406f-8c6a-a37eb87c1fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:45.732149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a18b95-5492-406f-8c6a-a37eb87c1fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:45.734900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a18b95-5492-406f-8c6a-a37eb87c1fd6 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:45.765981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 821/1646] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:46.059261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:46.613952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:47.340096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:47.345245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:47.421131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98269-3806-40b7-be16-5f31909b364f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:47.433871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:47.434590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:47.434862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:47.435125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] PATH_INFO: `/v3/users/32f6802c125e417791721c23e2e3c900` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:47.436673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f6802c125e417791721c23e2e3c900)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:47.438707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:47.817108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f6802c125e417791721c23e2e3c900'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:47.818210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f6802c125e417791721c23e2e3c900'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:47.849212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f6802c125e417791721c23e2e3c900'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:47.853058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f6802c125e417791721c23e2e3c900'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:47.972912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:48.019087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f6802c125e417791721c23e2e3c900'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:48.076832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 826/1647] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:34:47 2022] DELETE /identity/v3/users/32f6802c125e417791721c23e2e3c900 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:48.083019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:48.083798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:48.083798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:48.127289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:48.129166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:48.230184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 822/1648] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:48.307878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:48.311139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:48.494873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a22e54-846a-4235-aa5b-06b5a34ffeae admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:48.502489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:48.503947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:48.504538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:48.505354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:48.531251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:48.533577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:48.556090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 827/1649] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:34:49.695429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:49.699923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:49.754984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859ab8-1c28-4a33-a186-9f01a06232cf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:49.765320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:49.766789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:49.767458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:49.768796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] PATH_INFO: `/v3/projects/b89d036f42664c96a0ae9942d8bee18a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:49.773302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b89d036f42664c96a0ae9942d8bee18a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:49.778475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:49.859127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:50.078217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b89d036f42664c96a0ae9942d8bee18a', 'request_id': 'req-ac8fb187-7862-43f6-8f9c-1200fa547f85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:50.104335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b89d036f42664c96a0ae9942d8bee18a', 'request_id': 'req-ac8fb187-7862-43f6-8f9c-1200fa547f85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:50.121189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b89d036f42664c96a0ae9942d8bee18a', 'request_id': 'req-ac8fb187-7862-43f6-8f9c-1200fa547f85'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:50.266102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] The token cache is being invalidate because project b89d036f42664c96a0ae9942d8bee18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:34:50.269278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:50.274384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:50.282473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 823/1650] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:34:49 2022] DELETE /identity/v3/projects/b89d036f42664c96a0ae9942d8bee18a => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:50.289230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:50.325926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c487b-3baa-4796-a0e9-bd39fb5febdf service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:50.580572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:50.584776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:50.587905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:50.593908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] PATH_INFO: `/v3/users/2d43abf126d14efd8f2b6dfb8148b257` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:50.599274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d43abf126d14efd8f2b6dfb8148b257)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:50.610447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:51.077231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d43abf126d14efd8f2b6dfb8148b257'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:51.079560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d43abf126d14efd8f2b6dfb8148b257'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:51.098672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d43abf126d14efd8f2b6dfb8148b257'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:51.099682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d43abf126d14efd8f2b6dfb8148b257'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:51.213166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5019494e-2bb4-4aee-9508-48ba5f0a4ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d43abf126d14efd8f2b6dfb8148b257'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:51.233640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:51.252369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 828/1651] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:34:50 2022] DELETE /identity/v3/users/2d43abf126d14efd8f2b6dfb8148b257 => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:52.210967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:52.215019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:52.274745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fb187-7862-43f6-8f9c-1200fa547f85 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:52.493889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:52.497340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:52.498205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:52.499592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] PATH_INFO: `/v3/projects/85352dbb41424d14bab00dfc491b93c5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:52.502417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=85352dbb41424d14bab00dfc491b93c5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:52.510845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:52.594748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33773ae7-b05b-4463-b47e-54a9ac01d979 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:52.595466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33773ae7-b05b-4463-b47e-54a9ac01d979 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:52.595466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33773ae7-b05b-4463-b47e-54a9ac01d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:52.673230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-33773ae7-b05b-4463-b47e-54a9ac01d979 None None] MFA Rules not processed for user `c86ef757a41e4b9abb4fef7ef3c219b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:52.674472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-33773ae7-b05b-4463-b47e-54a9ac01d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:52.848129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85352dbb41424d14bab00dfc491b93c5', 'request_id': 'req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:52.867595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85352dbb41424d14bab00dfc491b93c5', 'request_id': 'req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:52.886209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85352dbb41424d14bab00dfc491b93c5', 'request_id': 'req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:52.930127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 829/1652] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:52 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:53.014322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] The token cache is being invalidate because project 85352dbb41424d14bab00dfc491b93c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:34:53.014617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:53.023142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e13ea3e0-fefd-4693-964c-d0c0cf150a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:53.025561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 824/1653] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:34:52 2022] DELETE /identity/v3/projects/85352dbb41424d14bab00dfc491b93c5 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:53.033137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5949e39-712f-4a70-9ea8-5130518b8551 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:53.033695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5949e39-712f-4a70-9ea8-5130518b8551 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:53.034205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5949e39-712f-4a70-9ea8-5130518b8551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:53.095778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a5949e39-712f-4a70-9ea8-5130518b8551 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:53.101247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a5949e39-712f-4a70-9ea8-5130518b8551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:53.318474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 830/1654] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:53.331706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358376f-2f31-48a7-8bd7-95da8a97ee7f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:53.331706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358376f-2f31-48a7-8bd7-95da8a97ee7f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:53.331706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358376f-2f31-48a7-8bd7-95da8a97ee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:53.388711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2358376f-2f31-48a7-8bd7-95da8a97ee7f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:53.390259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2358376f-2f31-48a7-8bd7-95da8a97ee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:53.410610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 825/1655] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:53.414396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5949e39-712f-4a70-9ea8-5130518b8551 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:53.430937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5949e39-712f-4a70-9ea8-5130518b8551 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:53.448188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:53.448598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:53.448877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:53.449314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:53.450247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:53.450551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:53.455773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:53.493840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 831/1656] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:53.495158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358376f-2f31-48a7-8bd7-95da8a97ee7f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:53.511832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358376f-2f31-48a7-8bd7-95da8a97ee7f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:53.547110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:53.548301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:53.553388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:53.555559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:53.560964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:53.570140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:53.896464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 826/1657] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:53 2022] POST /identity/v3/projects => generated 375 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:53.900909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:53.916535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fa914-728d-4139-91d9-a1ccea8c5f11 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:54.084740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:54.089588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:54.113803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:54.122407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:54.126765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:54.131136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:54.131250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:54.133817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:54.137364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:54.183132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:54.317405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 832/1658] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:53 2022] POST /identity/v3/users => generated 288 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:54.320829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:54.343183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bccbf-a299-489e-8e58-8483d50a753a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:54.368279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:54.368963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:54.369252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:54.370043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce86ca4-e23f-4786-a4a7-be9db87e3019 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:54.376912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:54.377744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:54.379437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:54.396973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:54.399501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:54.399791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:54.401862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] PATH_INFO: `/v3/projects/b624c98fbe764c4ca3374feab58a5900` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:54.408070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b624c98fbe764c4ca3374feab58a5900)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:54.414181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:54.433421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 833/1659] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:54.436437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:54.466132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03e948-2c67-44eb-87fb-b0732314ec3a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:54.472114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:54.472869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:54.473227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:54.473715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:54.478183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:54.480225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:54.525438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 834/1660] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:54.528191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:54.574953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0916b4b-0c53-43cc-b4b8-9edd65c3a752 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:54.593055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308908ce-74bc-4cc2-887d-e6b0bc1ca5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:54.594467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308908ce-74bc-4cc2-887d-e6b0bc1ca5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:54.601615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308908ce-74bc-4cc2-887d-e6b0bc1ca5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:54.602971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308908ce-74bc-4cc2-887d-e6b0bc1ca5dc admin admin] PATH_INFO: `/v3/projects/f68349a353f143a99a250b6489268598/users/b12a81ddb47146568ad5cf334a6d01eb/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:54.648168 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b624c98fbe764c4ca3374feab58a5900', 'request_id': 'req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:54.665186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308908ce-74bc-4cc2-887d-e6b0bc1ca5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68349a353f143a99a250b6489268598, user_id=b12a81ddb47146568ad5cf334a6d01eb, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:54.670789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308908ce-74bc-4cc2-887d-e6b0bc1ca5dc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:54.672092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b624c98fbe764c4ca3374feab58a5900', 'request_id': 'req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:54.691942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b624c98fbe764c4ca3374feab58a5900', 'request_id': 'req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:54.816698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] The token cache is being invalidate because project b624c98fbe764c4ca3374feab58a5900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:34:54.817052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:54.818147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f7785e89-a8e3-4a8c-93ac-bbc0d3145df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:54.823018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 827/1661] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:34:54 2022] DELETE /identity/v3/projects/b624c98fbe764c4ca3374feab58a5900 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:54.832967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73bab-985f-4ddc-ba30-c727adda926b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:54.832967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73bab-985f-4ddc-ba30-c727adda926b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:54.832967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73bab-985f-4ddc-ba30-c727adda926b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:54.856072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 835/1662] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:54 2022] PUT /identity/v3/projects/f68349a353f143a99a250b6489268598/users/b12a81ddb47146568ad5cf334a6d01eb/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:54.871609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:54.872234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:54.872800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:54.882997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-56a73bab-985f-4ddc-ba30-c727adda926b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:54.886936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-56a73bab-985f-4ddc-ba30-c727adda926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:55.002780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] MFA Rules not processed for user `b12a81ddb47146568ad5cf334a6d01eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:55.005288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:55.022099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 828/1663] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:55.035306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c953-5320-404d-960f-64e61847439a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:55.035588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c953-5320-404d-960f-64e61847439a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:55.035856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c953-5320-404d-960f-64e61847439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:55.082047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d146c953-5320-404d-960f-64e61847439a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:55.083443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d146c953-5320-404d-960f-64e61847439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:55.124111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 829/1664] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:55.130224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146c953-5320-404d-960f-64e61847439a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:55.166099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146c953-5320-404d-960f-64e61847439a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:55.193310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:55.194648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:55.195458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:55.199430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:55.200806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:34:55.202730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:55.212029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:55.229852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 836/1665] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:55.275805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:55.275998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 830/1666] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:34:55.282538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:55.285265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:55.330079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfbd89-a162-4db0-bef3-ebacf6b90539 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:55.345286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:55.346449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:55.347125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:55.347715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:55.348895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:55.361358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:55.556292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dceec-d17d-4145-9828-a3fe7fd1d35d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:55.584161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:55.585780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:55.585780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:55.586143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:55.603028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:55.608198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:55.611348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:55.620566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 831/1667] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:55 2022] POST /identity/v3/projects => generated 367 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:55.629833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:55.666237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 837/1668] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:55.880404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54691ae-ebd3-41ed-a3a6-1d4626aa8b85 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:55.906352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:55.906352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:55.906352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:55.906352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:55.908863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:55.908863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:56.060487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 832/1669] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:55 2022] POST /identity/v3/users => generated 284 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:56.062895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:56.075572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84870f82-e6f4-4ce5-ac09-87330223af0c service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:56.102690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2f81b-fe65-4d7f-85a7-a3fdf9ba7762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:56.104121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2f81b-fe65-4d7f-85a7-a3fdf9ba7762 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:56.105279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2f81b-fe65-4d7f-85a7-a3fdf9ba7762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:56.108148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2f81b-fe65-4d7f-85a7-a3fdf9ba7762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:56.111657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2f81b-fe65-4d7f-85a7-a3fdf9ba7762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:56.119705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2f81b-fe65-4d7f-85a7-a3fdf9ba7762 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:56.181204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:56.189835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 838/1670] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:56.189835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a60c45-bb9e-45b1-836c-5f1a6f046e2b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:56.189835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a60c45-bb9e-45b1-836c-5f1a6f046e2b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:56.189835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a60c45-bb9e-45b1-836c-5f1a6f046e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:56.221984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba80923-0ff9-4731-b196-f337285dde95 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:56.231071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:56.232417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:56.233012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:56.233590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:56.237508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:56.246394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:56.253502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:56.271920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-86a60c45-bb9e-45b1-836c-5f1a6f046e2b None None] MFA Rules not processed for user `3570472f8c2b42b480ec1d1220414bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:56.275317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-86a60c45-bb9e-45b1-836c-5f1a6f046e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:56.282608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 833/1671] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:56.282608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:56.286119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:56.331412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a51be-b3dc-4b50-a3bf-79af24ccfe39 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:56.351650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:56.353443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:56.359619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:56.359619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] PATH_INFO: `/v3/users/b12a81ddb47146568ad5cf334a6d01eb` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:56.359619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12a81ddb47146568ad5cf334a6d01eb)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:56.368897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:56.550427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 839/1672] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:56.553316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a60c45-bb9e-45b1-836c-5f1a6f046e2b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:56.589953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a60c45-bb9e-45b1-836c-5f1a6f046e2b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:56.604058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789f8b8-57e2-4343-97aa-7184fe84ba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:56.608943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789f8b8-57e2-4343-97aa-7184fe84ba4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:56.609429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789f8b8-57e2-4343-97aa-7184fe84ba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:56.609911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789f8b8-57e2-4343-97aa-7184fe84ba4b admin admin] PATH_INFO: `/v3/projects/f1c2b33924b7477bb2101c1c4ab09cc4/users/8d723edd48844d0c9e3ad7d35b2e3a7f/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:56.665451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789f8b8-57e2-4343-97aa-7184fe84ba4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c2b33924b7477bb2101c1c4ab09cc4, user_id=8d723edd48844d0c9e3ad7d35b2e3a7f, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:56.682098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789f8b8-57e2-4343-97aa-7184fe84ba4b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:56.844415 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12a81ddb47146568ad5cf334a6d01eb'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:56.847492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12a81ddb47146568ad5cf334a6d01eb'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:56.847492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12a81ddb47146568ad5cf334a6d01eb'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:56.930995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 840/1673] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:56 2022] PUT /identity/v3/projects/f1c2b33924b7477bb2101c1c4ab09cc4/users/8d723edd48844d0c9e3ad7d35b2e3a7f/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:56.933050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f02c1-94cd-40f7-a7e1-b05b873ceac9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:56.933589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f02c1-94cd-40f7-a7e1-b05b873ceac9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:56.934076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f02c1-94cd-40f7-a7e1-b05b873ceac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:56.979693 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12a81ddb47146568ad5cf334a6d01eb'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:57.021278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b65f02c1-94cd-40f7-a7e1-b05b873ceac9 None None] MFA Rules not processed for user `3570472f8c2b42b480ec1d1220414bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:57.023707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f0286a3d-2cd6-4f13-b994-eac206923612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12a81ddb47146568ad5cf334a6d01eb'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:34:57.036617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b65f02c1-94cd-40f7-a7e1-b05b873ceac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:57.114468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 834/1674] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:34:56 2022] DELETE /identity/v3/users/b12a81ddb47146568ad5cf334a6d01eb => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:57.117819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:57.118369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:57.118864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:57.199231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 841/1675] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:57.200822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:57.201221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:57.201476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:57.238656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] MFA Rules not processed for user `8d723edd48844d0c9e3ad7d35b2e3a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:57.240950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:57.277067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:57.281393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:57.448668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:57.457415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 842/1676] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:57.467329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:57.469056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:57.556238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 835/1677] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:57.557767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:57.568501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:57.674530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:57.809316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70295ed9-1786-4e61-86c4-268852bf4197 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:57.824820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:57.826364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:57.826823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:57.827230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:57.838238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e669658-6e1e-47e5-8140-2113a9510bc2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:57.852190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:57.854334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:57.854471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:57.856014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:57.856595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:57.857223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:57.860786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:57.885673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 836/1678] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:57.887374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:57.889058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:57.939243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d839c-6e4f-4760-b626-2d57a24a8b52 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:57.955091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec170e81-2ea0-4b7a-82b8-ddae10c60e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:57.957080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec170e81-2ea0-4b7a-82b8-ddae10c60e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:57.958115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec170e81-2ea0-4b7a-82b8-ddae10c60e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:57.959431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec170e81-2ea0-4b7a-82b8-ddae10c60e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:57.962534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec170e81-2ea0-4b7a-82b8-ddae10c60e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:57.976994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec170e81-2ea0-4b7a-82b8-ddae10c60e35 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:58.006446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:58.011888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:58.043587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 843/1679] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:58.278633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 837/1680] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:34:57 2022] POST /identity/v3/projects => generated 367 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:58.282915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:58.289021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436421e-f394-4994-a28b-e05508c0b097 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:58.289501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436421e-f394-4994-a28b-e05508c0b097 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:58.289970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436421e-f394-4994-a28b-e05508c0b097 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:58.297302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 838/1681] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:34:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:34:58.312491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cffde2-6282-470d-b273-d990be4a4a49 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:58.333497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565f86-bb0d-4a08-9287-aa9f8efc28e5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:58.336309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565f86-bb0d-4a08-9287-aa9f8efc28e5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:58.336309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565f86-bb0d-4a08-9287-aa9f8efc28e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:58.472802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5e565f86-bb0d-4a08-9287-aa9f8efc28e5 None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:58.477525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5e565f86-bb0d-4a08-9287-aa9f8efc28e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:58.646581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7070f-9a01-497c-b4af-a5e1575f52db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:58.648347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7070f-9a01-497c-b4af-a5e1575f52db admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:58.649298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7070f-9a01-497c-b4af-a5e1575f52db admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:58.649875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7070f-9a01-497c-b4af-a5e1575f52db admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:58.655154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7070f-9a01-497c-b4af-a5e1575f52db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:58.667557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7070f-9a01-497c-b4af-a5e1575f52db admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:58.801220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 839/1682] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:34:58.805581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2d6b-914a-4f63-aa21-89e0c83b3bfc None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:58.805947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2d6b-914a-4f63-aa21-89e0c83b3bfc None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:58.806507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2d6b-914a-4f63-aa21-89e0c83b3bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:58.873843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ac5e2d6b-914a-4f63-aa21-89e0c83b3bfc None None] MFA Rules not processed for user `34b1955a937f4053967cf82cedd87a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:58.875996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ac5e2d6b-914a-4f63-aa21-89e0c83b3bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:58.943750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 844/1683] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:34:58 2022] POST /identity/v3/users => generated 290 bytes in 660 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:34:58.955812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d200d-1194-4b74-b057-5242879aafb2 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:58.959216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d200d-1194-4b74-b057-5242879aafb2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:58.959216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d200d-1194-4b74-b057-5242879aafb2 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:58.999875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 845/1684] 10.222.0.49 () {54 vars in 961 bytes} [Wed Aug 24 14:34:58 2022] GET /identity => generated 270 bytes in 45 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:34:59.003705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:59.010172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d200d-1194-4b74-b057-5242879aafb2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:59.066817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d200d-1194-4b74-b057-5242879aafb2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:59.079471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 840/1685] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:59.081901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:59.082539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:59.083072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:59.099713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:59.120063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:59.121116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:59.121675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:59.124916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:59.165417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:59.216021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:34:59.217342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:59.245000 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 841/1686] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Aug 24 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:34:59.246097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:59.252616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:34:59.254260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 846/1687] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:34:59.260862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:59.306808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efd97f-2dd2-48db-bff6-da1bac70adc3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:59.339377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:59.340705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:59.341188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:59.341666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] PATH_INFO: `/v3/projects/a71a72db31a24854aa47f6b339806ef4/users/ed414249700e47ebbd3db1b6b24be120/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:59.433883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71a72db31a24854aa47f6b339806ef4, user_id=ed414249700e47ebbd3db1b6b24be120, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:59.450739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:59.532887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26176ced-1e3c-4d34-9244-b4c9ae70f826 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:59.572108 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:59.573208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:59.573710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:59.577631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:34:59.619876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:34:59.632275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:34:59.657740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 847/1688] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:34:59 2022] PUT /identity/v3/projects/a71a72db31a24854aa47f6b339806ef4/users/ed414249700e47ebbd3db1b6b24be120/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:34:59.657740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:59.686843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 842/1689] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:34:59.689590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:34:59.716378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:34:59.778615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5aef9-827d-492c-9cf3-92bbaee409fb service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:59.972864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cb48-2996-4aad-a04b-00a2b982108f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:34:59.975567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e39642-a058-424d-b168-d0ad9ad40d7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:34:59.976550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e39642-a058-424d-b168-d0ad9ad40d7d service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:34:59.977022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e39642-a058-424d-b168-d0ad9ad40d7d service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:34:59.977592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e39642-a058-424d-b168-d0ad9ad40d7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.009918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e39642-a058-424d-b168-d0ad9ad40d7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:00.011974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e39642-a058-424d-b168-d0ad9ad40d7d service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:00.024273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:00.024900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:00.025181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:00.025463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.026161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:00.030787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:00.033058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 848/1690] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:00.034171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f830d-967e-498b-955b-821a8611ddb0 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:00.035190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f830d-967e-498b-955b-821a8611ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:00.035190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f830d-967e-498b-955b-821a8611ddb0 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.037167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 849/1691] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 24 14:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:35:00.047794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:00.049360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:00.049360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.059386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 843/1692] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:00.063965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:00.092177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057a0e5-e27d-4869-a527-93e3ba3a9b92 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:00.103635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e862fb-8086-4784-9c22-536f273de588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:00.105610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e862fb-8086-4784-9c22-536f273de588 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:00.107365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e862fb-8086-4784-9c22-536f273de588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:00.108100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e862fb-8086-4784-9c22-536f273de588 admin admin] PATH_INFO: `/v3/projects/a71a72db31a24854aa47f6b339806ef4/users/ed414249700e47ebbd3db1b6b24be120/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.117947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e862fb-8086-4784-9c22-536f273de588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71a72db31a24854aa47f6b339806ef4, user_id=ed414249700e47ebbd3db1b6b24be120, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:00.125198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e862fb-8086-4784-9c22-536f273de588 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:00.166017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] MFA Rules not processed for user `48364ccf19434c4c9c09b4e5ce4bfc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:00.176123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 844/1693] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:00 2022] PUT /identity/v3/projects/a71a72db31a24854aa47f6b339806ef4/users/ed414249700e47ebbd3db1b6b24be120/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:00.177210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:00.180584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:00.184542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:00.184542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.297016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] MFA Rules not processed for user `ed414249700e47ebbd3db1b6b24be120`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:00.299331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:00.300154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 850/1694] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:00.344935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:00.347234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:00.386449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe1511-1aaa-4dfb-8031-5cb0fae4be94 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:00.391342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3104a-fd67-4e00-8f3b-3046635ddfdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:00.391995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3104a-fd67-4e00-8f3b-3046635ddfdc service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:00.392276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3104a-fd67-4e00-8f3b-3046635ddfdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:00.392561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3104a-fd67-4e00-8f3b-3046635ddfdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.407158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3104a-fd67-4e00-8f3b-3046635ddfdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:00.415204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3104a-fd67-4e00-8f3b-3046635ddfdc service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:00.456073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:00.477606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 851/1695] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:35:00.482253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:00.482780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:00.486211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:00.650215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] MFA Rules not processed for user `7b946ecd87434ad6a00890887d234281`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:00.659909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:00.664113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 845/1696] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:00.665821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:00.669707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:00.927820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 852/1697] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:00.927820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:00.927820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:01.002169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:01.045227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d36ae-2419-4f1e-a7e1-7c7853509265 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:01.067994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:01.071075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:01.077749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:01.077749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] PATH_INFO: `/v3/projects/f68349a353f143a99a250b6489268598` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:01.077749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68349a353f143a99a250b6489268598)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:01.085954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:01.150036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318137c-bda6-4f84-b69d-56a14542eec6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:01.162753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:01.166722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:01.170090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:01.171086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:01.202795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:01.205893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:01.234518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 853/1698] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:01.240098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:01.324368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68349a353f143a99a250b6489268598', 'request_id': 'req-e8c11742-43e3-47fb-bcb0-15adea4b34e7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:01.334978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68349a353f143a99a250b6489268598', 'request_id': 'req-e8c11742-43e3-47fb-bcb0-15adea4b34e7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:01.352892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68349a353f143a99a250b6489268598', 'request_id': 'req-e8c11742-43e3-47fb-bcb0-15adea4b34e7'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:01.480522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0e614-2fb4-49d2-b229-de92c5bea574 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:01.485873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] The token cache is being invalidate because project f68349a353f143a99a250b6489268598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:01.486490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:01.487645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:01.490845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 846/1699] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:00 2022] DELETE /identity/v3/projects/f68349a353f143a99a250b6489268598 => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:01.492896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:01.496506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:01.598210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1d908-19d2-48e6-9501-d84219eb6532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:01.599680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d908-19d2-48e6-9501-d84219eb6532 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:01.600228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d908-19d2-48e6-9501-d84219eb6532 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:01.601237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d908-19d2-48e6-9501-d84219eb6532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:01.603921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c8c1d908-19d2-48e6-9501-d84219eb6532 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:01.670466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c11742-43e3-47fb-bcb0-15adea4b34e7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:01.677903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:01.680798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:01.681436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:01.681922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:01.686567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:01.733861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:01.735773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:01.739939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1d908-19d2-48e6-9501-d84219eb6532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:01.743791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1d908-19d2-48e6-9501-d84219eb6532 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:01.759535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 854/1700] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:01.761630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b998447-6ce7-4c7c-81ea-3690d9a5cbef None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:01.762108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b998447-6ce7-4c7c-81ea-3690d9a5cbef None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:01.762441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b998447-6ce7-4c7c-81ea-3690d9a5cbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:01.786025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0b998447-6ce7-4c7c-81ea-3690d9a5cbef None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:01.787434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0b998447-6ce7-4c7c-81ea-3690d9a5cbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:01.997608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:01.997608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:01.998336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 855/1701] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:01.998336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce5a84-ad48-448f-b9e3-c1a9b3daf72d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:01.998336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce5a84-ad48-448f-b9e3-c1a9b3daf72d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:01.998336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce5a84-ad48-448f-b9e3-c1a9b3daf72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:02.019654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 847/1702] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:02.094374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-05ce5a84-ad48-448f-b9e3-c1a9b3daf72d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:02.097128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-05ce5a84-ad48-448f-b9e3-c1a9b3daf72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:02.124290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 856/1703] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:02.128751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:02.143072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3559d8-6143-4731-b094-c15a44667e86 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:02.195110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:02.195753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:02.199891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:02.200374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:02.202607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:02.202936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:02.216904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:02.274580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce5a84-ad48-448f-b9e3-c1a9b3daf72d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:02.277003 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 848/1704] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:35:02.345616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce5a84-ad48-448f-b9e3-c1a9b3daf72d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:02.378161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:02.378161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:02.378161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:02.378161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:02.378494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:02.385406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:02.791808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 857/1705] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:02 2022] POST /identity/v3/projects => generated 363 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:02.795065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:02.813266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc05f82-9402-4170-91b6-883379bce45c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:02.982346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:02.983951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:02.984322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:02.990594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:02.990594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:02.990594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:03.027151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:03.030820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:03.106806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca43e00-d92e-4886-9745-1a806952a2dc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:03.119222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 849/1706] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:02 2022] POST /identity/v3/users => generated 282 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:03.123622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:03.134576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:03.135592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:03.136064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:03.136724 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] PATH_INFO: `/v3/users/efb54407effd46a1816e10b164848bf2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:03.148647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb54407effd46a1816e10b164848bf2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:03.151163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c108486-f7ef-4f2e-9e33-1fa000b873ce admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:03.152919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:03.164771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:03.165407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:03.165703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:03.168210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:03.170971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:03.172604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:03.222637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 850/1707] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:03.224533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:03.258120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31d460-25b1-4fb3-8126-761f3d27e432 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:03.281968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:03.282662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:03.282953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:03.283643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:03.283976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:03.286461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:03.328928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 851/1708] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:03.336773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:03.436190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940c00-54b1-418c-b9b2-83fb09f4b2be admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:03.474934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e53545-02de-40c2-bc2b-08e040e4b072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:03.491844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e53545-02de-40c2-bc2b-08e040e4b072 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:03.491844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e53545-02de-40c2-bc2b-08e040e4b072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:03.491844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e53545-02de-40c2-bc2b-08e040e4b072 admin admin] PATH_INFO: `/v3/projects/5fe9a0cd1cde4c51bbf8048251d5e2f5/users/53b1c2444ada4df99f43384f9b9336df/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:03.615363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb54407effd46a1816e10b164848bf2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:03.620118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e53545-02de-40c2-bc2b-08e040e4b072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe9a0cd1cde4c51bbf8048251d5e2f5, user_id=53b1c2444ada4df99f43384f9b9336df, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:03.621120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb54407effd46a1816e10b164848bf2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:03.669764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e53545-02de-40c2-bc2b-08e040e4b072 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:03.718853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb54407effd46a1816e10b164848bf2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:03.723854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb54407effd46a1816e10b164848bf2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:03.889615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 852/1709] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:03 2022] PUT /identity/v3/projects/5fe9a0cd1cde4c51bbf8048251d5e2f5/users/53b1c2444ada4df99f43384f9b9336df/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:03.905424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:03.905424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:03.905424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:03.933119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e76b6654-ed52-46f7-bf7d-7943d23ee8d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb54407effd46a1816e10b164848bf2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:04.059251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 858/1710] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:03 2022] DELETE /identity/v3/users/efb54407effd46a1816e10b164848bf2 => generated 0 bytes in 1033 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:04.061865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:04.067231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:04.067753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:04.133010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] MFA Rules not processed for user `c888090dd9f246a79c6be54faf480ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:04.144600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:04.213585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] MFA Rules not processed for user `53b1c2444ada4df99f43384f9b9336df`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:04.215760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:04.306404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:04.344688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:04.398166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 853/1711] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:04.398166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:04.399254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:04.446763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 859/1712] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:04.446763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:04.451743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:04.652278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15b892-d54a-414d-aa55-4069fca81e72 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:04.665409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:04.666340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:04.666762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:04.667175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:04.693100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:04.696136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:04.715520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 860/1713] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:04.719395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:04.722919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:04.733198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1826e5-c3fd-4d4c-85f8-6b74b4156cd7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:04.763388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44f41e-c201-4108-843c-0678b93ea2d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:04.768928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44f41e-c201-4108-843c-0678b93ea2d1 service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:04.770697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44f41e-c201-4108-843c-0678b93ea2d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:04.771207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44f41e-c201-4108-843c-0678b93ea2d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:04.779865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-af44f41e-c201-4108-843c-0678b93ea2d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:04.842979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44f41e-c201-4108-843c-0678b93ea2d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:04.849471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44f41e-c201-4108-843c-0678b93ea2d1 service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:04.883549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 854/1714] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:04.886391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1169-1c42-48f6-abb3-d96407378541 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:04.886851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1169-1c42-48f6-abb3-d96407378541 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:04.887302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1169-1c42-48f6-abb3-d96407378541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:04.888444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90a87c-5a4e-4f4a-afed-891402397748 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:04.917688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:04.918377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:04.918739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:04.919074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:04.919771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:04.921819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:04.929828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9a7b1169-1c42-48f6-abb3-d96407378541 None None] MFA Rules not processed for user `3570472f8c2b42b480ec1d1220414bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:04.932818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9a7b1169-1c42-48f6-abb3-d96407378541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:05.118853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 855/1715] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:05.152312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 861/1716] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:04 2022] POST /identity/v3/projects => generated 363 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:05.163435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b1169-1c42-48f6-abb3-d96407378541 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:05.198374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b1169-1c42-48f6-abb3-d96407378541 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:05.523875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:05.539182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:05.539661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:05.540309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:05.541510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:05.550279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:05.724446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 856/1717] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:05 2022] POST /identity/v3/users => generated 288 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:05.725598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:05.746748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf73fd6-68e6-4e5d-b8b9-f4302b0ec4c3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:05.781485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:05.787428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:05.788460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:05.795052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:05.797091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:05.808079 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:05.830262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:05.900304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 862/1718] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:05.906295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:05.926311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca6145-e3b5-4504-ac6e-6fd02d06fd45 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:05.947662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dabb8-0ef4-457d-b2e9-e08081012a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:05.950960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dabb8-0ef4-457d-b2e9-e08081012a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:05.951293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dabb8-0ef4-457d-b2e9-e08081012a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:05.955719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dabb8-0ef4-457d-b2e9-e08081012a3c admin admin] PATH_INFO: `/v3/projects/7990cdd9f1844b6d800c3a397a2e7173/users/3fcc7415edb5441d8c4e506874447931/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:05.997212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dabb8-0ef4-457d-b2e9-e08081012a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7990cdd9f1844b6d800c3a397a2e7173, user_id=3fcc7415edb5441d8c4e506874447931, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:06.000999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dabb8-0ef4-457d-b2e9-e08081012a3c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:06.134967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 857/1719] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:05 2022] PUT /identity/v3/projects/7990cdd9f1844b6d800c3a397a2e7173/users/3fcc7415edb5441d8c4e506874447931/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:06.136204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:06.150459 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ad02d-281d-482c-98ec-7a8a526a9b8e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:06.295193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:06.296183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:06.296183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:06.297431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:06.299400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:06.304877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be71-199a-4e48-abb5-fc8e01c98df3 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:06.304877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be71-199a-4e48-abb5-fc8e01c98df3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:06.304877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be71-199a-4e48-abb5-fc8e01c98df3 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:06.305866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 858/1720] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 24 14:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:35:06.307887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:06.321180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:06.321776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:06.322581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:06.390686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 863/1721] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:06.392001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:06.406059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] MFA Rules not processed for user `48364ccf19434c4c9c09b4e5ce4bfc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:06.409139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:06.436050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd365190-1fb7-4860-902e-09676e49f4d4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:06.452065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb62a-25f0-4e0f-8c46-6e906c4989f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:06.453248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb62a-25f0-4e0f-8c46-6e906c4989f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:06.457312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb62a-25f0-4e0f-8c46-6e906c4989f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:06.457932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb62a-25f0-4e0f-8c46-6e906c4989f5 admin admin] PATH_INFO: `/v3/projects/7990cdd9f1844b6d800c3a397a2e7173/users/3fcc7415edb5441d8c4e506874447931/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:06.464302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbb62a-25f0-4e0f-8c46-6e906c4989f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7990cdd9f1844b6d800c3a397a2e7173, user_id=3fcc7415edb5441d8c4e506874447931, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:06.473032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbb62a-25f0-4e0f-8c46-6e906c4989f5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:06.512367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 864/1722] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:06 2022] PUT /identity/v3/projects/7990cdd9f1844b6d800c3a397a2e7173/users/3fcc7415edb5441d8c4e506874447931/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:06.517630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:06.517946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:06.518216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:06.647315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 859/1723] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:35:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:06.648959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:06.651866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] MFA Rules not processed for user `3fcc7415edb5441d8c4e506874447931`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:06.653112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:06.657344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:06.711287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918e59d-1823-4358-b3c2-a53ed3d01903 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:06.903581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:06.904319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:06.904627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:06.918227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] PATH_INFO: `/v3/projects/7be521d6de8a4275808a41cba50c36e1` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:06.918227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be521d6de8a4275808a41cba50c36e1)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:06.918227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:06.920899 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 865/1724] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:06.930695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:06.935923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:07.095339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:07.121922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1829b4-96a9-4a34-b958-2287ed0600ad None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:07.132573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:07.137458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:07.137458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:07.137458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:07.159155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:07.173498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:07.196200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be521d6de8a4275808a41cba50c36e1', 'request_id': 'req-f68f56d4-1217-432c-b624-9e5b9c385cca'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:07.211550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 866/1725] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:07.212512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be521d6de8a4275808a41cba50c36e1', 'request_id': 'req-f68f56d4-1217-432c-b624-9e5b9c385cca'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:07.213246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:07.219494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:07.248026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be521d6de8a4275808a41cba50c36e1', 'request_id': 'req-f68f56d4-1217-432c-b624-9e5b9c385cca'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:07.335836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d5699-e5a9-4628-abf0-2b68c38a7319 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:07.408189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] The token cache is being invalidate because project 7be521d6de8a4275808a41cba50c36e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:07.408189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:07.408189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:07.420037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 860/1726] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:06 2022] DELETE /identity/v3/projects/7be521d6de8a4275808a41cba50c36e1 => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:07.425246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:07.431405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:07.481324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68236f-fd9e-44c1-9d6e-777e72e24827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:07.482361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68236f-fd9e-44c1-9d6e-777e72e24827 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:07.482797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68236f-fd9e-44c1-9d6e-777e72e24827 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:07.488072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68236f-fd9e-44c1-9d6e-777e72e24827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:07.488827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9c68236f-fd9e-44c1-9d6e-777e72e24827 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:07.724288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f56d4-1217-432c-b624-9e5b9c385cca admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:07.753485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68236f-fd9e-44c1-9d6e-777e72e24827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:07.755111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:07.756317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:07.758122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:07.758765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] PATH_INFO: `/v3/users/c86ef757a41e4b9abb4fef7ef3c219b6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:07.766518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86ef757a41e4b9abb4fef7ef3c219b6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:07.780005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:07.784333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68236f-fd9e-44c1-9d6e-777e72e24827 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:07.825086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 867/1727] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:08.185198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86ef757a41e4b9abb4fef7ef3c219b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:08.186589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86ef757a41e4b9abb4fef7ef3c219b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:08.192203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86ef757a41e4b9abb4fef7ef3c219b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:08.206949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:08.346264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86ef757a41e4b9abb4fef7ef3c219b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:08.346380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f1db-d4d5-4591-a223-e17d027b8904 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:08.346380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f1db-d4d5-4591-a223-e17d027b8904 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:08.346380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f1db-d4d5-4591-a223-e17d027b8904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:08.347692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86ef757a41e4b9abb4fef7ef3c219b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:08.409408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 861/1728] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:07 2022] DELETE /identity/v3/users/c86ef757a41e4b9abb4fef7ef3c219b6 => generated 0 bytes in 983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:08.409408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:08.412276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:08.455186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bba0f1db-d4d5-4591-a223-e17d027b8904 None None] MFA Rules not processed for user `e84d3ad291644b84b204b0b3102a095a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:08.457703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bba0f1db-d4d5-4591-a223-e17d027b8904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:09.207694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba4285-a222-4e58-aefa-55615cb8145b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:09.207694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:09.207694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:09.207694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:09.213921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:09.213921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:09.216594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 868/1729] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:09.217729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:09.217729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:09.219276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1b31b-8538-4980-b90d-edb3a0b85898 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:09.219276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1b31b-8538-4980-b90d-edb3a0b85898 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:09.219276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1b31b-8538-4980-b90d-edb3a0b85898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:09.219884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 862/1730] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:09.220321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2df1b31b-8538-4980-b90d-edb3a0b85898 None None] MFA Rules not processed for user `e84d3ad291644b84b204b0b3102a095a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:09.220321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2df1b31b-8538-4980-b90d-edb3a0b85898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:09.220321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 869/1731] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:09.223523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:09.223523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:09.223523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226a09-37b4-4f9c-8490-6f2652ce7d55 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:09.224402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:09.224402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:09.224402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:09.224402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:09.225034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:09.225034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:09.225034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 863/1732] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:09.420673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4bb25-0730-4443-9f87-a27e85448613 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:09.421860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4bb25-0730-4443-9f87-a27e85448613 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:09.422543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4bb25-0730-4443-9f87-a27e85448613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:09.526961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fbf4bb25-0730-4443-9f87-a27e85448613 None None] MFA Rules not processed for user `34b1955a937f4053967cf82cedd87a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:09.533015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fbf4bb25-0730-4443-9f87-a27e85448613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:09.694622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 870/1733] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:10.576136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:10.578972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:10.789150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:10.817952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e405b6-b1e7-42fe-8fa7-4673ddc2b897 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:10.833627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:10.835236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:10.837124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:10.838326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] PATH_INFO: `/v3/projects/51421852ecf449dbab34d7cfa974abae` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:10.842096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51421852ecf449dbab34d7cfa974abae)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:10.845806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:11.140110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51421852ecf449dbab34d7cfa974abae', 'request_id': 'req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:11.169247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51421852ecf449dbab34d7cfa974abae', 'request_id': 'req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:11.196885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51421852ecf449dbab34d7cfa974abae', 'request_id': 'req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:11.336649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] The token cache is being invalidate because project 51421852ecf449dbab34d7cfa974abae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:11.337105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:11.344543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d409b7ad-58c2-4de0-b256-bbdd1f4aee69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:11.362461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 864/1734] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:10 2022] DELETE /identity/v3/projects/51421852ecf449dbab34d7cfa974abae => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:11.364134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf62a94-9d53-4fbf-bffe-40c3dc253cc6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:11.364793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf62a94-9d53-4fbf-bffe-40c3dc253cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:11.365386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf62a94-9d53-4fbf-bffe-40c3dc253cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:11.444812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-adf62a94-9d53-4fbf-bffe-40c3dc253cc6 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:11.446907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-adf62a94-9d53-4fbf-bffe-40c3dc253cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:11.632647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 871/1735] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:11.648217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ea3f7-9595-45bf-a89e-c2e69ad6981e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:11.652198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ea3f7-9595-45bf-a89e-c2e69ad6981e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:11.652198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ea3f7-9595-45bf-a89e-c2e69ad6981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:11.770364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-cf2ea3f7-9595-45bf-a89e-c2e69ad6981e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:11.779126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cf2ea3f7-9595-45bf-a89e-c2e69ad6981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:11.867247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 865/1736] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:11.867941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf62a94-9d53-4fbf-bffe-40c3dc253cc6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:11.880432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf62a94-9d53-4fbf-bffe-40c3dc253cc6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:11.910034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:11.910034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:11.910034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:11.910034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:11.910681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:11.910681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:11.910681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:11.955629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 872/1737] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:35:11.965186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ea3f7-9595-45bf-a89e-c2e69ad6981e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:12.020531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ea3f7-9595-45bf-a89e-c2e69ad6981e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:12.086184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:12.091801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:12.093908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:12.101429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:12.101429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:12.114604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:12.329274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 866/1738] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:11 2022] POST /identity/v3/projects => generated 371 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:12.330933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:12.363937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8dfb8-9b54-4c70-9581-7719f5061c69 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:12.708657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:12.712493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:12.800347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:12.802804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:12.809398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:12.809976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:12.811100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:12.825618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:13.009399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 873/1739] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:12 2022] POST /identity/v3/users => generated 286 bytes in 680 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:13.032977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:13.131591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a49ec-4ac7-4fc7-8e2f-376ee9a50a24 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:13.180510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a04be-c63a-4b0d-a50e-8608a1e0cf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:13.181648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a04be-c63a-4b0d-a50e-8608a1e0cf15 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:13.182214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a04be-c63a-4b0d-a50e-8608a1e0cf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:13.186980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a04be-c63a-4b0d-a50e-8608a1e0cf15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:13.188408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a04be-c63a-4b0d-a50e-8608a1e0cf15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:13.201163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5a23-238c-4793-a146-1cc27a4f89a3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:13.212547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a04be-c63a-4b0d-a50e-8608a1e0cf15 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:13.228005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:13.228899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:13.229188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:13.229839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] PATH_INFO: `/v3/users/8d723edd48844d0c9e3ad7d35b2e3a7f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:13.241097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d723edd48844d0c9e3ad7d35b2e3a7f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:13.243490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:13.313599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 874/1740] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:13.328858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:13.329416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:13.329918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:13.456495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:13.458479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:13.560270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d723edd48844d0c9e3ad7d35b2e3a7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:13.560926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d723edd48844d0c9e3ad7d35b2e3a7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:13.561547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d723edd48844d0c9e3ad7d35b2e3a7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:13.738581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d723edd48844d0c9e3ad7d35b2e3a7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:13.753372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3642a6cf-95de-44f0-95be-2fde6a815d4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d723edd48844d0c9e3ad7d35b2e3a7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:13.789782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 875/1741] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:35:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:13.790858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:13.792635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:13.800380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 867/1742] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:12 2022] DELETE /identity/v3/users/8d723edd48844d0c9e3ad7d35b2e3a7f => generated 0 bytes in 1093 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:13.801440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:13.801720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:13.801985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:13.835677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914b20a-3379-42b0-ac70-1ef8b22c72eb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:13.892790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] MFA Rules not processed for user `7b946ecd87434ad6a00890887d234281`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:13.894422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:13.946943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7db65-bc12-4c6b-845f-0ceda9524b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:13.947730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7db65-bc12-4c6b-845f-0ceda9524b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:13.948192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7db65-bc12-4c6b-845f-0ceda9524b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:13.948565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7db65-bc12-4c6b-845f-0ceda9524b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:13.949472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7db65-bc12-4c6b-845f-0ceda9524b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:13.951409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7db65-bc12-4c6b-845f-0ceda9524b14 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:14.004048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 876/1743] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:14.006363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eb606-94fa-422e-9058-b4c2ca36ac6f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:14.007449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eb606-94fa-422e-9058-b4c2ca36ac6f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:14.007855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eb606-94fa-422e-9058-b4c2ca36ac6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:14.043408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f97eb606-94fa-422e-9058-b4c2ca36ac6f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:14.048981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f97eb606-94fa-422e-9058-b4c2ca36ac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:14.064378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 877/1744] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:14.066429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eb606-94fa-422e-9058-b4c2ca36ac6f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:14.090540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eb606-94fa-422e-9058-b4c2ca36ac6f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:14.090685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 868/1745] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:14.092398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72408731-4130-4772-bd7e-44f825646e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:14.093151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72408731-4130-4772-bd7e-44f825646e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:14.093441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72408731-4130-4772-bd7e-44f825646e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:14.117704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72408731-4130-4772-bd7e-44f825646e07 admin admin] PATH_INFO: `/v3/projects/e7af817492f44ad9b80060db5a5e3ab7/users/7084143e092045758a231ce4298e52f1/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:14.120147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:14.122488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:14.123583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:14.184773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72408731-4130-4772-bd7e-44f825646e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7af817492f44ad9b80060db5a5e3ab7, user_id=7084143e092045758a231ce4298e52f1, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:14.189815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72408731-4130-4772-bd7e-44f825646e07 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:14.276815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d149c8-ff93-4e79-8814-a584f93d0114 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:14.288616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:14.289757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:14.290253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:14.291286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:14.321234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:14.321234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:14.337675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 869/1746] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:14.409147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:14.412482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:14.424976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 878/1747] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:14 2022] PUT /identity/v3/projects/e7af817492f44ad9b80060db5a5e3ab7/users/7084143e092045758a231ce4298e52f1/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:14.426965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:14.445526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355c16a-fe42-49fe-a463-c9deecce4fb9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:14.446021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355c16a-fe42-49fe-a463-c9deecce4fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:14.446456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355c16a-fe42-49fe-a463-c9deecce4fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:14.547547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7355c16a-fe42-49fe-a463-c9deecce4fb9 None None] MFA Rules not processed for user `7084143e092045758a231ce4298e52f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:14.550248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7355c16a-fe42-49fe-a463-c9deecce4fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:14.729114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b33d-0273-4d41-bd47-d7723ecc41c7 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:14.891886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 879/1748] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:14.909235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355c16a-fe42-49fe-a463-c9deecce4fb9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:14.975995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:14.975995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:14.975995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:14.975995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:15.164056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355c16a-fe42-49fe-a463-c9deecce4fb9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:15.195541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:15.203529 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:15.204072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:15.208803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:15.211784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:15.213033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:15.215074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:15.227970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:15.264719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 870/1749] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:35:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 856 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:35:15.317401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:15.321445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:15.390622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e46205-f72e-4900-ac1b-eee4874d75fd service swift] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:15.430846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:15.431528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:15.431932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:15.432095 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] PATH_INFO: `/v3/users/53b1c2444ada4df99f43384f9b9336df` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:15.435360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53b1c2444ada4df99f43384f9b9336df)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:15.437330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:15.496294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 880/1750] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:14 2022] POST /identity/v3/projects => generated 371 bytes in 599 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:15.505786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:15.858819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c2f39-3f7f-46d9-9f76-bd7561e4f92f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:15.892979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5670fd-f2b6-4647-a227-16767e2df09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:15.906481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5670fd-f2b6-4647-a227-16767e2df09c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:15.906481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5670fd-f2b6-4647-a227-16767e2df09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:15.906481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5670fd-f2b6-4647-a227-16767e2df09c admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:15.906481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5670fd-f2b6-4647-a227-16767e2df09c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:15.912555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5670fd-f2b6-4647-a227-16767e2df09c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:15.959077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53b1c2444ada4df99f43384f9b9336df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:15.962845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53b1c2444ada4df99f43384f9b9336df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:15.967064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53b1c2444ada4df99f43384f9b9336df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:15.998909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:16.105895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53b1c2444ada4df99f43384f9b9336df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:16.118930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 881/1751] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:15 2022] POST /identity/v3/users => generated 292 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:16.120784 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d558c8-9eab-4976-bd49-94b1bfed01ad None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:16.122487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d558c8-9eab-4976-bd49-94b1bfed01ad None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:16.123901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53b1c2444ada4df99f43384f9b9336df'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:16.125641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d558c8-9eab-4976-bd49-94b1bfed01ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:16.257986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 871/1752] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:15 2022] DELETE /identity/v3/users/53b1c2444ada4df99f43384f9b9336df => generated 0 bytes in 942 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:16.263925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:16.269302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-00d558c8-9eab-4976-bd49-94b1bfed01ad None None] MFA Rules not processed for user `48364ccf19434c4c9c09b4e5ce4bfc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:16.273318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:16.276003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-00d558c8-9eab-4976-bd49-94b1bfed01ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:16.552186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 882/1753] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 24 14:35:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:16.555690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c55340-fec4-4b05-98f9-f9e1f4f63c87 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:16.556128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c55340-fec4-4b05-98f9-f9e1f4f63c87 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:16.558326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c55340-fec4-4b05-98f9-f9e1f4f63c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:16.590629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47704be8-8ef5-4d86-a387-b4c74f34b6a3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:16.608639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:16.609686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:16.610266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:16.610842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:16.614919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:16.617590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:16.619240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-16c55340-fec4-4b05-98f9-f9e1f4f63c87 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:16.624369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-16c55340-fec4-4b05-98f9-f9e1f4f63c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:16.646896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 872/1754] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:16.649507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 883/1755] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:16.650771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:16.655268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c55340-fec4-4b05-98f9-f9e1f4f63c87 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:16.662538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:16.695552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c55340-fec4-4b05-98f9-f9e1f4f63c87 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:16.721689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdfb7d-b35f-415b-9df4-ef08768fb3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:16.721689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdfb7d-b35f-415b-9df4-ef08768fb3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:16.721689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdfb7d-b35f-415b-9df4-ef08768fb3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:16.721689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdfb7d-b35f-415b-9df4-ef08768fb3d9 admin admin] PATH_INFO: `/v3/projects/bc85bbed5d324a298b03a3cdab99c40c/users/c69112c520d3477a87f730832c5baeb8/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:16.748224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ef51-cdd1-4660-a7e5-935de096900d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:16.777887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:16.779110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdfb7d-b35f-415b-9df4-ef08768fb3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc85bbed5d324a298b03a3cdab99c40c, user_id=c69112c520d3477a87f730832c5baeb8, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:16.779778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:16.781101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:16.781941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:16.795684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdfb7d-b35f-415b-9df4-ef08768fb3d9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:16.841372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:16.847250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:16.908559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 873/1756] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:16.910704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:16.933872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 884/1757] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:16 2022] PUT /identity/v3/projects/bc85bbed5d324a298b03a3cdab99c40c/users/c69112c520d3477a87f730832c5baeb8/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:16.939295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95727b22-4c9c-4c45-a500-b21207dec033 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:16.939295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95727b22-4c9c-4c45-a500-b21207dec033 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:16.939295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95727b22-4c9c-4c45-a500-b21207dec033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:16.974522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef40c2e-22b2-4bc9-a350-21cc1863fcc8 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:16.991747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-95727b22-4c9c-4c45-a500-b21207dec033 None None] MFA Rules not processed for user `c888090dd9f246a79c6be54faf480ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:16.996782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-95727b22-4c9c-4c45-a500-b21207dec033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:17.271683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:17.273399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:17.275543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:17.276021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:17.315962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 885/1758] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:17.317627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95727b22-4c9c-4c45-a500-b21207dec033 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:17.516481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:17.518099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:17.545022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 874/1759] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:17.630018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95727b22-4c9c-4c45-a500-b21207dec033 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:17.638926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:17.646890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:17.647721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:17.650543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:17.651357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:17.656384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:17.695294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 886/1760] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:17.701249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:17.716230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7fd05-a5e4-4d01-851c-c0354fee5b92 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:17.743766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:17.746832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:17.748510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efb1c2-dd6f-4702-a11c-b40857429f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:17.764397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efb1c2-dd6f-4702-a11c-b40857429f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:17.775104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efb1c2-dd6f-4702-a11c-b40857429f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:17.775104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efb1c2-dd6f-4702-a11c-b40857429f15 admin admin] PATH_INFO: `/v3/projects/bc85bbed5d324a298b03a3cdab99c40c/users/c69112c520d3477a87f730832c5baeb8/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:17.783995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efb1c2-dd6f-4702-a11c-b40857429f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc85bbed5d324a298b03a3cdab99c40c, user_id=c69112c520d3477a87f730832c5baeb8, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:17.797769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cb6d6-c16e-483c-8086-095c9232b703 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:17.815075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efb1c2-dd6f-4702-a11c-b40857429f15 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:17.822277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:17.829453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:17.829453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:17.829453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] PATH_INFO: `/v3/projects/f1c2b33924b7477bb2101c1c4ab09cc4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:17.837383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c2b33924b7477bb2101c1c4ab09cc4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:17.840764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 14:35:17.840764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 14:35:17.840764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: Aug 24 14:35:17.846748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:17.868736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 875/1761] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:17 2022] PUT /identity/v3/projects/bc85bbed5d324a298b03a3cdab99c40c/users/c69112c520d3477a87f730832c5baeb8/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:17.875152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:17.875152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:17.875152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:17.970786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] MFA Rules not processed for user `c69112c520d3477a87f730832c5baeb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:17.972922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:18.208079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 876/1762] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:18.223884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:18.235463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:18.266955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c2b33924b7477bb2101c1c4ab09cc4', 'request_id': 'req-0aff94ac-b8ca-469e-aca2-b1a9da48639a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:18.298791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c2b33924b7477bb2101c1c4ab09cc4', 'request_id': 'req-0aff94ac-b8ca-469e-aca2-b1a9da48639a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:18.423906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c2b33924b7477bb2101c1c4ab09cc4', 'request_id': 'req-0aff94ac-b8ca-469e-aca2-b1a9da48639a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:18.448412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4bde4-aa05-4d13-925b-31ee09ac01c8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:18.526578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] The token cache is being invalidate because project f1c2b33924b7477bb2101c1c4ab09cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:18.532402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:18.532402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:18.533474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:18.542847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 887/1763] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:17 2022] DELETE /identity/v3/projects/f1c2b33924b7477bb2101c1c4ab09cc4 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:18.555557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:18.558495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:18.663349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:18.664658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:18.664841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:18.666058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:18.668490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:18.722645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:18.832029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff94ac-b8ca-469e-aca2-b1a9da48639a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:18.838633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:18.839343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:18.839679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:18.839954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] PATH_INFO: `/v3/users/ed414249700e47ebbd3db1b6b24be120` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:18.842934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed414249700e47ebbd3db1b6b24be120)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:18.844942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:19.073745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:19.077938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:19.108381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed414249700e47ebbd3db1b6b24be120'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:19.113538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed414249700e47ebbd3db1b6b24be120'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:19.149461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 877/1764] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 927 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:19.170848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed414249700e47ebbd3db1b6b24be120'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:19.190594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed414249700e47ebbd3db1b6b24be120'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:19.292337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed414249700e47ebbd3db1b6b24be120'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:19.362113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 888/1765] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:18 2022] DELETE /identity/v3/users/ed414249700e47ebbd3db1b6b24be120 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:19.510886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:19.513406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:19.616804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804514a-3df5-4467-93b5-45705647c98e service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:19.886220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192cc2-aafe-489e-b856-830269f94819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:19.887873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192cc2-aafe-489e-b856-830269f94819 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:19.888375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192cc2-aafe-489e-b856-830269f94819 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:19.888856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192cc2-aafe-489e-b856-830269f94819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:19.908719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d7192cc2-aafe-489e-b856-830269f94819 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:20.045741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7192cc2-aafe-489e-b856-830269f94819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:20.045741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7192cc2-aafe-489e-b856-830269f94819 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:20.045741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 878/1766] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:20.991743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:20.994245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:21.356185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3de55d-0d6b-4f9c-8775-7962b238ac77 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:21.394842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:21.394842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:21.394842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:21.394842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] PATH_INFO: `/v3/projects/5fe9a0cd1cde4c51bbf8048251d5e2f5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:21.395650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe9a0cd1cde4c51bbf8048251d5e2f5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:21.399560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:21.443760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:21.721383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe9a0cd1cde4c51bbf8048251d5e2f5', 'request_id': 'req-1068aae3-cf3c-490e-903d-5bdd55e472ce'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:21.767283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe9a0cd1cde4c51bbf8048251d5e2f5', 'request_id': 'req-1068aae3-cf3c-490e-903d-5bdd55e472ce'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:21.863959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb605a0-030f-48ab-bef6-34a5923eae06 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:21.868962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb605a0-030f-48ab-bef6-34a5923eae06 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:21.868962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb605a0-030f-48ab-bef6-34a5923eae06 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:21.868962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 879/1767] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:35:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:35:21.964289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe9a0cd1cde4c51bbf8048251d5e2f5', 'request_id': 'req-1068aae3-cf3c-490e-903d-5bdd55e472ce'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.032352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:22.032833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:22.033652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:22.043653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] The token cache is being invalidate because project 5fe9a0cd1cde4c51bbf8048251d5e2f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:22.044507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.054607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.057559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 889/1768] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:20 2022] DELETE /identity/v3/projects/5fe9a0cd1cde4c51bbf8048251d5e2f5 => generated 0 bytes in 1067 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:22.062767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:22.064931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:22.158814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:22.170972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:22.259886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068aae3-cf3c-490e-903d-5bdd55e472ce admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:22.271949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:22.276147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:22.278040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:22.280972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] PATH_INFO: `/v3/users/3fcc7415edb5441d8c4e506874447931` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:22.286642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fcc7415edb5441d8c4e506874447931)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:22.292212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:22.299939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:22.465371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 880/1769] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:35:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:22.752480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fcc7415edb5441d8c4e506874447931'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.753813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fcc7415edb5441d8c4e506874447931'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.784682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fcc7415edb5441d8c4e506874447931'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.785676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fcc7415edb5441d8c4e506874447931'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.901099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fcc7415edb5441d8c4e506874447931'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:22.962059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 890/1770] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:22 2022] DELETE /identity/v3/users/3fcc7415edb5441d8c4e506874447931 => generated 0 bytes in 900 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:23.144135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:23.146674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:23.171056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a5494-3cd2-4fe1-aa45-addc3221dcdd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:23.457877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3bc14-e6a3-46df-afd0-41c67c7e864d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:23.464102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bc14-e6a3-46df-afd0-41c67c7e864d service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:23.467689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bc14-e6a3-46df-afd0-41c67c7e864d service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:23.470798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bc14-e6a3-46df-afd0-41c67c7e864d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:23.483300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-71f3bc14-e6a3-46df-afd0-41c67c7e864d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:23.529227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3bc14-e6a3-46df-afd0-41c67c7e864d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:23.532139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3bc14-e6a3-46df-afd0-41c67c7e864d service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:23.540350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:23.584527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 881/1771] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:24.280610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:24.282559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:24.553011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacb632-a921-4f49-9101-58bf5725ac91 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:24.568143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:24.568872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:24.569191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:24.572370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] PATH_INFO: `/v3/projects/a71a72db31a24854aa47f6b339806ef4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:24.575311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71a72db31a24854aa47f6b339806ef4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:24.581059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:24.858328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71a72db31a24854aa47f6b339806ef4', 'request_id': 'req-9ab5dc0f-b5d1-4864-93ed-bed3475db903'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:24.903452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71a72db31a24854aa47f6b339806ef4', 'request_id': 'req-9ab5dc0f-b5d1-4864-93ed-bed3475db903'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:24.991609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71a72db31a24854aa47f6b339806ef4', 'request_id': 'req-9ab5dc0f-b5d1-4864-93ed-bed3475db903'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:25.025245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] The token cache is being invalidate because project a71a72db31a24854aa47f6b339806ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:25.025532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:25.028736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:25.040535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0d24-35da-4c2a-ab90-920e99c3d9ed None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:25.041579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0d24-35da-4c2a-ab90-920e99c3d9ed None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:25.042114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0d24-35da-4c2a-ab90-920e99c3d9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:25.045319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 891/1772] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:24 2022] DELETE /identity/v3/projects/a71a72db31a24854aa47f6b339806ef4 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:25.088658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0f9b0d24-35da-4c2a-ab90-920e99c3d9ed None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:25.093159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0f9b0d24-35da-4c2a-ab90-920e99c3d9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:25.213745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:25.216042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:25.414876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 882/1773] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:25.425721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321fd8a-4132-47c1-858a-65355cf222fd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:25.426630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321fd8a-4132-47c1-858a-65355cf222fd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:25.427204 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321fd8a-4132-47c1-858a-65355cf222fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:25.512390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f321fd8a-4132-47c1-858a-65355cf222fd None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:25.515015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f321fd8a-4132-47c1-858a-65355cf222fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:25.527037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5dc0f-b5d1-4864-93ed-bed3475db903 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:25.535243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 883/1774] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:25.541889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321fd8a-4132-47c1-858a-65355cf222fd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:25.558820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:25.561942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:25.562470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:25.562998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] PATH_INFO: `/v3/projects/7990cdd9f1844b6d800c3a397a2e7173` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:25.569056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:25.574650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7990cdd9f1844b6d800c3a397a2e7173)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:25.574893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321fd8a-4132-47c1-858a-65355cf222fd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:25.576535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:25.593445 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:25.596733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:25.600129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:25.608799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:25.608799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:25.608799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:25.618531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:25.697427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 884/1775] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:35:25.700272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:25.729520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41d010-f347-4c01-aa13-ee10ded07235 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:25.761478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:25.765030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:25.765488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:25.766028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:25.767288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:25.777435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:25.905728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7990cdd9f1844b6d800c3a397a2e7173', 'request_id': 'req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:25.922295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7990cdd9f1844b6d800c3a397a2e7173', 'request_id': 'req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:25.952323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 885/1776] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:25 2022] POST /identity/v3/projects => generated 399 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:25.958968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:26.041962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7990cdd9f1844b6d800c3a397a2e7173', 'request_id': 'req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:26.091636 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] The token cache is being invalidate because project 7990cdd9f1844b6d800c3a397a2e7173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:26.095148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:26.096500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a01fc4d7-b16a-4ce7-9078-c57099a1d5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:26.103294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 892/1777] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:25 2022] DELETE /identity/v3/projects/7990cdd9f1844b6d800c3a397a2e7173 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:26.112757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed0d3-459d-4df1-b734-38ecc2b815e8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.113394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed0d3-459d-4df1-b734-38ecc2b815e8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.113962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed0d3-459d-4df1-b734-38ecc2b815e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:26.195086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7f9ed0d3-459d-4df1-b734-38ecc2b815e8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:26.195086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f9ed0d3-459d-4df1-b734-38ecc2b815e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:26.218520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8461d-d7da-4e5a-9eae-388dbd21f259 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:26.254119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 893/1778] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:26.263331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c02d-f7e1-4f05-8245-e531c55fa134 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.263960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c02d-f7e1-4f05-8245-e531c55fa134 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.268830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c02d-f7e1-4f05-8245-e531c55fa134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:26.291176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:26.303342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.304778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.305378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:26.309784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:26.327528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:26.396740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3f55c02d-f7e1-4f05-8245-e531c55fa134 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:26.405256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3f55c02d-f7e1-4f05-8245-e531c55fa134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:26.477291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 894/1779] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:26.479087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c02d-f7e1-4f05-8245-e531c55fa134 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:26.514751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c02d-f7e1-4f05-8245-e531c55fa134 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:26.542433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:26.543523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.544019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.549487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:26.551028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:26.551584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:26.558078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:26.624255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 895/1780] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:35:26.636888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:26.643747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 886/1781] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:25 2022] POST /identity/v3/users => generated 300 bytes in 685 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:26.656502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:26.660647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:26.669395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10a77f-55b6-41f3-8996-eefba1e1c936 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:26.686317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:26.688953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.691049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.691049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:26.692998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:26.697418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:26.722505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef0268-2099-46ac-929d-0f5514492e86 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:26.732051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:26.732051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.732051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.732051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:26.732443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:26.732443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:26.756392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 887/1782] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:26.761957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:26.793310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10646-ecd3-4344-b32d-e3aef0765868 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:26.807007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:26.808522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.808807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.809146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:26.810889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:26.813876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:26.859186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 888/1783] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:26.879009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:26.919923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaaad4-9820-4fac-a8bb-2940b889fd7e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:26.953308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81742b-9107-41af-b853-554dd4fb9a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:26.954428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81742b-9107-41af-b853-554dd4fb9a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:26.954883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81742b-9107-41af-b853-554dd4fb9a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:26.955331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81742b-9107-41af-b853-554dd4fb9a36 admin admin] PATH_INFO: `/v3/projects/7969649ab2574ad5aaebae137da7c556/users/f1431529f5e04ad6818bb0b3aba05e93/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:27.020154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 896/1784] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:26 2022] POST /identity/v3/projects => generated 379 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:27.025440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:27.043030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81742b-9107-41af-b853-554dd4fb9a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7969649ab2574ad5aaebae137da7c556, user_id=f1431529f5e04ad6818bb0b3aba05e93, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:27.049027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81742b-9107-41af-b853-554dd4fb9a36 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:27.158435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 889/1785] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:26 2022] PUT /identity/v3/projects/7969649ab2574ad5aaebae137da7c556/users/f1431529f5e04ad6818bb0b3aba05e93/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:27.178864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948ffc9-55d9-4a66-b58f-f88e819b365c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:27.179310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948ffc9-55d9-4a66-b58f-f88e819b365c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:27.179691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948ffc9-55d9-4a66-b58f-f88e819b365c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:27.188594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:27.265214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1948ffc9-55d9-4a66-b58f-f88e819b365c None None] MFA Rules not processed for user `f1431529f5e04ad6818bb0b3aba05e93`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:27.267248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1948ffc9-55d9-4a66-b58f-f88e819b365c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:27.300128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf441c83-6c36-4b1e-b79d-051f32ed4a02 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:27.350926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:27.485774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:27.487378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:27.487564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 890/1786] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:27.488230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:27.488922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:27.490511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:27.494998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:27.497257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948ffc9-55d9-4a66-b58f-f88e819b365c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:27.523012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948ffc9-55d9-4a66-b58f-f88e819b365c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:27.543663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:27.545333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:27.545788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:27.550253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:27.550253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:27.551150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:27.731345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 897/1787] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:27 2022] POST /identity/v3/users => generated 290 bytes in 710 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:27.757119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:27.778414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 891/1788] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:27 2022] POST /identity/v3/projects => generated 401 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:27.782732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:27.789231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221774a-cb4e-49b1-803c-9eea283b05f3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:27.882733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:27.883539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:27.883829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:27.884366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:27.886085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:27.889065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:27.912269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 898/1789] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:35:27.913799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:27.957932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819580-8956-41aa-ad2b-572bd1bcef8e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:27.969576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:27.970310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:27.970615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:27.970863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:27.972097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:27.973882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:27.975110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68610ba7-40a3-4ff4-aa1c-86556f352d3a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:27.991895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:27.995378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:27.995378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:27.995378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:27.995378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:28.000359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 899/1790] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:28.002078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:28.004258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:28.027909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3ee54-771d-4fe0-9192-7b87d8eb354a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:28.043000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2533c-6b81-4eb6-bea9-1c775f222a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:28.047036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2533c-6b81-4eb6-bea9-1c775f222a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:28.047036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2533c-6b81-4eb6-bea9-1c775f222a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:28.047036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2533c-6b81-4eb6-bea9-1c775f222a3b admin admin] PATH_INFO: `/v3/projects/ba0f8d74965f40b4ae5915e89fe2fddb/users/71b5d2ab35b94185b912037bf2c27edb/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:28.123614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2533c-6b81-4eb6-bea9-1c775f222a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0f8d74965f40b4ae5915e89fe2fddb, user_id=71b5d2ab35b94185b912037bf2c27edb, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:28.128865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2533c-6b81-4eb6-bea9-1c775f222a3b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:28.173773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 892/1791] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:27 2022] POST /identity/v3/users => generated 307 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:28.174842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:28.201276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3f6c7-d69a-4730-a90d-d5b9292152c2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:28.217715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:28.219002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:28.219687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:28.220311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:28.221579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:28.226012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:28.276311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 893/1792] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:28.285158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:28.302816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 900/1793] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:28 2022] PUT /identity/v3/projects/ba0f8d74965f40b4ae5915e89fe2fddb/users/71b5d2ab35b94185b912037bf2c27edb/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:28.315867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55a724-ee7d-4713-af38-4516e4dda79e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:28.316155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55a724-ee7d-4713-af38-4516e4dda79e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:28.316440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55a724-ee7d-4713-af38-4516e4dda79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:28.325931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc390-0c79-4b4a-849f-240bcbc983c3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:28.407884 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9c55a724-ee7d-4713-af38-4516e4dda79e None None] MFA Rules not processed for user `71b5d2ab35b94185b912037bf2c27edb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:28.410718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9c55a724-ee7d-4713-af38-4516e4dda79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:28.500564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:28.501546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:28.501832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:28.502160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] PATH_INFO: `/v3/projects/c085b88bdbe34f2591f5f68e5c05c3f1/users/27e16716ded7427bbe2cac1da45007ee/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:28.527721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c085b88bdbe34f2591f5f68e5c05c3f1, user_id=27e16716ded7427bbe2cac1da45007ee, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:28.531784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:28.568377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 901/1794] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:28.573234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55a724-ee7d-4713-af38-4516e4dda79e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:28.585822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55a724-ee7d-4713-af38-4516e4dda79e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:28.591610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:28.592269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:28.592697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:28.593062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:28.593667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:28.595702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:28.800780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 894/1795] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:28 2022] PUT /identity/v3/projects/c085b88bdbe34f2591f5f68e5c05c3f1/users/27e16716ded7427bbe2cac1da45007ee/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:28.805318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:28.856393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 902/1796] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:28 2022] POST /identity/v3/projects => generated 381 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:35:28.863036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:29.090197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce99a0b-2e6a-4983-8167-d7d4a597c2bf admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:29.109761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:29.119739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.119739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.121395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb297f-0b06-4b38-aaf3-383a668c097f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:29.121680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.121680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:29.124710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:29.133529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:29.134791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.135225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.135648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.136498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:29.139219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:29.167914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 895/1797] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:29.174721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:29.194937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfd90e-b70b-44f5-a2e2-852f5ed9d5c8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:29.200093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27cec1-b77e-4084-8630-06b2fd889707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:29.201114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27cec1-b77e-4084-8630-06b2fd889707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.201565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27cec1-b77e-4084-8630-06b2fd889707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.202781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27cec1-b77e-4084-8630-06b2fd889707 admin admin] PATH_INFO: `/v3/projects/c085b88bdbe34f2591f5f68e5c05c3f1/users/27e16716ded7427bbe2cac1da45007ee/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.206793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27cec1-b77e-4084-8630-06b2fd889707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c085b88bdbe34f2591f5f68e5c05c3f1, user_id=27e16716ded7427bbe2cac1da45007ee, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:29.210957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27cec1-b77e-4084-8630-06b2fd889707 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:29.236808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 896/1798] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:29 2022] PUT /identity/v3/projects/c085b88bdbe34f2591f5f68e5c05c3f1/users/27e16716ded7427bbe2cac1da45007ee/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:29.248808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.249115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.249464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.308096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] MFA Rules not processed for user `27e16716ded7427bbe2cac1da45007ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:29.310175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:29.362801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 903/1799] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:28 2022] POST /identity/v3/users => generated 297 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:29.377374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:29.401363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a073b9-3305-45ed-97cc-edac1f382463 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:29.512430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4c2d1-cbfd-4341-8da3-61ebf979d982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:29.513231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4c2d1-cbfd-4341-8da3-61ebf979d982 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.513629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4c2d1-cbfd-4341-8da3-61ebf979d982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.514059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4c2d1-cbfd-4341-8da3-61ebf979d982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.514923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4c2d1-cbfd-4341-8da3-61ebf979d982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:29.516989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4c2d1-cbfd-4341-8da3-61ebf979d982 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:29.523089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 897/1800] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:29 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:29.537871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:29.543629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 904/1801] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:35:29.543801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:29.543882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4065c-e068-4d5d-8ce1-9efb0d49da7c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.543882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4065c-e068-4d5d-8ce1-9efb0d49da7c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.543882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4065c-e068-4d5d-8ce1-9efb0d49da7c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.544490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 905/1802] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:35:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:35:29.547650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4065c-e068-4d5d-8ce1-9efb0d49da7c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:29.579584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4065c-e068-4d5d-8ce1-9efb0d49da7c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:29.613623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:29.617586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.618291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.619083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] PATH_INFO: `/v3/projects/076db353e5434c518733b044e3c9ae29/users/54a3ccd1e258485e8a87105ffc2a3105/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.660127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] RBAC: Authorizing `identity:create_grant(project_id=076db353e5434c518733b044e3c9ae29, user_id=54a3ccd1e258485e8a87105ffc2a3105, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:29.665857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:29.679174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3768f-2c2a-4175-9d87-88c52fff89f2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:29.685491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397e545-8b9f-4dde-b2ab-caa75ae480fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:29.686504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397e545-8b9f-4dde-b2ab-caa75ae480fa service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.687055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397e545-8b9f-4dde-b2ab-caa75ae480fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.687522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397e545-8b9f-4dde-b2ab-caa75ae480fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.705262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397e545-8b9f-4dde-b2ab-caa75ae480fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:29.706960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397e545-8b9f-4dde-b2ab-caa75ae480fa service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:29.725233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 898/1803] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:29.728241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.728740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.729222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.763580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:29.766170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:29.805012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 906/1804] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:29 2022] PUT /identity/v3/projects/076db353e5434c518733b044e3c9ae29/users/54a3ccd1e258485e8a87105ffc2a3105/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:29.808207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:29.962972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 899/1805] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:35:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:29.975733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620d43-ab55-4001-be13-33ae0811e64e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:29.980101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:29.980745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:29.981029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:29.981302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:29.982198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:29.983937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:29.986401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:29.988033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:30.003617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22c2bb-4e04-47e6-8126-ad3ffb9d14a4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:30.011760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 907/1806] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:35:30.017608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:30.046133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a1768-0ffc-40a2-bf62-351431d0729d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:30.053224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:30.054719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:30.055269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:30.055660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] PATH_INFO: `/v3/projects/076db353e5434c518733b044e3c9ae29/users/54a3ccd1e258485e8a87105ffc2a3105/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:30.081556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076db353e5434c518733b044e3c9ae29, user_id=54a3ccd1e258485e8a87105ffc2a3105, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:30.088980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:30.127289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 908/1807] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:30 2022] PUT /identity/v3/projects/076db353e5434c518733b044e3c9ae29/users/54a3ccd1e258485e8a87105ffc2a3105/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:30.130773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:30.201308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623ae73-9d41-4956-a121-1c8364eb78c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:30.203228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623ae73-9d41-4956-a121-1c8364eb78c8 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:30.204305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623ae73-9d41-4956-a121-1c8364eb78c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:30.205127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623ae73-9d41-4956-a121-1c8364eb78c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:30.236567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623ae73-9d41-4956-a121-1c8364eb78c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:30.239913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623ae73-9d41-4956-a121-1c8364eb78c8 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:30.263873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 900/1808] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:30.266504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221cb8-de4d-4bca-b0b6-90f99a1b5910 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:30.267015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221cb8-de4d-4bca-b0b6-90f99a1b5910 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:30.267518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221cb8-de4d-4bca-b0b6-90f99a1b5910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:30.314955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a2b67-98f5-443f-8b4c-5239cc57cf07 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:30.315462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:30.316181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:30.316181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:30.316181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:30.316181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:30.357721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-72221cb8-de4d-4bca-b0b6-90f99a1b5910 None None] MFA Rules not processed for user `54a3ccd1e258485e8a87105ffc2a3105`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:30.360899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-72221cb8-de4d-4bca-b0b6-90f99a1b5910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:30.418025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:30.432501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:30.454665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 909/1809] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:30.624346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 901/1810] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:30.643377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:30.645479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:30.669653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9605642-d770-418d-8053-7b96744f8272 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:30.683750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:30.685815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:30.686719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:30.689801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:30.718048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:30.723899 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:30.764836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 910/1811] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:35:30.790066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:31.068723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221cb8-de4d-4bca-b0b6-90f99a1b5910 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:31.084361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221cb8-de4d-4bca-b0b6-90f99a1b5910 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:31.107391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6cbb1-2b48-452d-a7ac-60871cfda00d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:31.108036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6cbb1-2b48-452d-a7ac-60871cfda00d service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:31.108317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6cbb1-2b48-452d-a7ac-60871cfda00d service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:31.108593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6cbb1-2b48-452d-a7ac-60871cfda00d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:31.306155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6cbb1-2b48-452d-a7ac-60871cfda00d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:31.309416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6cbb1-2b48-452d-a7ac-60871cfda00d service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:31.388111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 902/1812] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:35:32.142390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:33.500043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:34.587889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:34.968955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:36.512420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:36.698429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2ccfa-6efe-4d83-aa21-12ce4877fb62 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:36.703287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:36.703969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:36.708905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:36.708905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] PATH_INFO: `/v3/users/71b5d2ab35b94185b912037bf2c27edb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:36.708905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b5d2ab35b94185b912037bf2c27edb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:36.708905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:37.085801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b5d2ab35b94185b912037bf2c27edb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:37.087370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b5d2ab35b94185b912037bf2c27edb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:37.110406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b5d2ab35b94185b912037bf2c27edb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:37.112384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b5d2ab35b94185b912037bf2c27edb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:37.242595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b5d2ab35b94185b912037bf2c27edb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:37.331209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 911/1813] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:36 2022] DELETE /identity/v3/users/71b5d2ab35b94185b912037bf2c27edb => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:37.338750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:37.339489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:37.340205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:37.426251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:37.430989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:37.604023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 903/1814] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:37.640498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:37.643030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:37.794028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac9d1a-30fa-47cf-a79d-0bb8221c1fbe admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:37.800022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:37.818190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:37.818190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:37.818190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:37.822326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:37.824092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:37.847567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 912/1815] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:38.213572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:38.323647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:38.325443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:38.338908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6dcdd-e7bb-484f-9abd-91535712c688 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:38.345978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:38.346943 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:38.347520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:38.347836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] PATH_INFO: `/v3/users/f1431529f5e04ad6818bb0b3aba05e93` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:38.349334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1431529f5e04ad6818bb0b3aba05e93)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:38.352630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:38.775333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1431529f5e04ad6818bb0b3aba05e93'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:38.776612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1431529f5e04ad6818bb0b3aba05e93'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:38.777948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1431529f5e04ad6818bb0b3aba05e93'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:38.991745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1431529f5e04ad6818bb0b3aba05e93'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:39.017379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f05c8cd7-cb87-401f-94fd-223a4dffcfdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1431529f5e04ad6818bb0b3aba05e93'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:39.054593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:39.055343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 904/1816] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:38 2022] DELETE /identity/v3/users/f1431529f5e04ad6818bb0b3aba05e93 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:39.058733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:39.064055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:39.065651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:39.066512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:39.102815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:39.104394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:39.120092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20b178-fec3-4c64-b815-a2c5a1a11c15 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:39.274022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 905/1817] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:39.286360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:39.298579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:39.304787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:39.304787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:39.304787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:39.304787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] PATH_INFO: `/v3/projects/ba0f8d74965f40b4ae5915e89fe2fddb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:39.306983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0f8d74965f40b4ae5915e89fe2fddb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:39.309319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:39.410704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd9243-eca2-4a27-9a23-ecd76929be44 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:39.428961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774ce20-30d6-4590-84a0-57af16ff8091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:39.430288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ce20-30d6-4590-84a0-57af16ff8091 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:39.431003 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ce20-30d6-4590-84a0-57af16ff8091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:39.431534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ce20-30d6-4590-84a0-57af16ff8091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:39.444050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774ce20-30d6-4590-84a0-57af16ff8091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:39.446964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774ce20-30d6-4590-84a0-57af16ff8091 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:39.459409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 906/1818] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:39.462977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e2951-8e6f-4fb7-8e27-a050581ff4ef None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:39.463740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e2951-8e6f-4fb7-8e27-a050581ff4ef None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:39.465703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e2951-8e6f-4fb7-8e27-a050581ff4ef None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:39.471742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 907/1819] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:35:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:35:39.473182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:39.473641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:39.473924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:39.500660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:39.502564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:39.533732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:39.585546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0f8d74965f40b4ae5915e89fe2fddb', 'request_id': 'req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:39.608738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0f8d74965f40b4ae5915e89fe2fddb', 'request_id': 'req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:39.665311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 908/1820] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:35:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:39.687708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:39.690982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:39.710876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413af2f-96c8-4010-b293-6ac165c6424f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:39.795886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0f8d74965f40b4ae5915e89fe2fddb', 'request_id': 'req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:39.804375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:39.805028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:39.805352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:39.805727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:39.818214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] The token cache is being invalidate because project ba0f8d74965f40b4ae5915e89fe2fddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:39.818490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:39.818680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:39.820720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:39.820922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:39.826712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 913/1821] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:39 2022] DELETE /identity/v3/projects/ba0f8d74965f40b4ae5915e89fe2fddb => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:39.828535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:39.830568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:39.836161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 909/1822] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:35:39.913095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f39ab8-57fd-4277-8ad1-861d9c6e9bb5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:39.918124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:39.918788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:39.919065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:39.919338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] PATH_INFO: `/v3/users/54a3ccd1e258485e8a87105ffc2a3105` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:39.920513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a3ccd1e258485e8a87105ffc2a3105)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:39.922511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:40.175370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a3ccd1e258485e8a87105ffc2a3105'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.176894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a3ccd1e258485e8a87105ffc2a3105'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.194380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a3ccd1e258485e8a87105ffc2a3105'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.196158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a3ccd1e258485e8a87105ffc2a3105'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.326927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a3ccd1e258485e8a87105ffc2a3105'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.354586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 914/1823] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:39 2022] DELETE /identity/v3/users/54a3ccd1e258485e8a87105ffc2a3105 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:40.370110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:40.382122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:40.400792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe227d-71de-40a3-8797-55a17d109810 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:40.500534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:40.501647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:40.502172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:40.502650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] PATH_INFO: `/v3/projects/7969649ab2574ad5aaebae137da7c556` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:40.504874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7969649ab2574ad5aaebae137da7c556)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:40.509480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:40.706805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7969649ab2574ad5aaebae137da7c556', 'request_id': 'req-88f58671-a5dc-4f15-815a-8e45fde123ac'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.720620 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7969649ab2574ad5aaebae137da7c556', 'request_id': 'req-88f58671-a5dc-4f15-815a-8e45fde123ac'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.736094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7969649ab2574ad5aaebae137da7c556', 'request_id': 'req-88f58671-a5dc-4f15-815a-8e45fde123ac'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.976801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] The token cache is being invalidate because project 7969649ab2574ad5aaebae137da7c556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:40.978214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.980315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:40.986249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 910/1824] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:40 2022] DELETE /identity/v3/projects/7969649ab2574ad5aaebae137da7c556 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:40.992683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:40.996389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:41.061499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d74d4-9186-45fb-a06e-89b1d457b9b7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:41.319317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:41.320327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:41.320873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:41.321410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] PATH_INFO: `/v3/users/27e16716ded7427bbe2cac1da45007ee` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:41.323639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e16716ded7427bbe2cac1da45007ee)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:41.336294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:42.793098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:42.803688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:42.851287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f58671-a5dc-4f15-815a-8e45fde123ac admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:42.881420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:42.881420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:42.881420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:42.881420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] PATH_INFO: `/v3/projects/076db353e5434c518733b044e3c9ae29` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:42.884666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=076db353e5434c518733b044e3c9ae29)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:42.884666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:42.949500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e16716ded7427bbe2cac1da45007ee'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:42.952535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e16716ded7427bbe2cac1da45007ee'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:42.976099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e16716ded7427bbe2cac1da45007ee'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:42.977073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e16716ded7427bbe2cac1da45007ee'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:43.086042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4c4b501-5182-40a8-85b0-cab33fb36c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e16716ded7427bbe2cac1da45007ee'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:43.130702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076db353e5434c518733b044e3c9ae29', 'request_id': 'req-ff0da2f1-32b7-4bab-8103-273c9c5230ab'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:43.142719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076db353e5434c518733b044e3c9ae29', 'request_id': 'req-ff0da2f1-32b7-4bab-8103-273c9c5230ab'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:43.166708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076db353e5434c518733b044e3c9ae29', 'request_id': 'req-ff0da2f1-32b7-4bab-8103-273c9c5230ab'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:43.196250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 915/1825] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:40 2022] DELETE /identity/v3/users/27e16716ded7427bbe2cac1da45007ee => generated 0 bytes in 2187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:43.291524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] The token cache is being invalidate because project 076db353e5434c518733b044e3c9ae29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:43.292035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:43.293239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ff0da2f1-32b7-4bab-8103-273c9c5230ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:43.298054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 911/1826] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:42 2022] DELETE /identity/v3/projects/076db353e5434c518733b044e3c9ae29 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:43.305507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360a744-7b77-463b-a647-4f60a6f0a8da None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:43.306350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360a744-7b77-463b-a647-4f60a6f0a8da None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:43.307089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360a744-7b77-463b-a647-4f60a6f0a8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:43.356534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:43.358959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6360a744-7b77-463b-a647-4f60a6f0a8da None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:43.361886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6360a744-7b77-463b-a647-4f60a6f0a8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:43.486350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 916/1827] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:43.494906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0fc2f-bb26-4bde-bdab-b9a0c7bb2d67 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:43.495519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0fc2f-bb26-4bde-bdab-b9a0c7bb2d67 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:43.496104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0fc2f-bb26-4bde-bdab-b9a0c7bb2d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:43.533845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0ac0fc2f-bb26-4bde-bdab-b9a0c7bb2d67 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:43.536414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0ac0fc2f-bb26-4bde-bdab-b9a0c7bb2d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:43.567633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 912/1828] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:43.571240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360a744-7b77-463b-a647-4f60a6f0a8da None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:43.592794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360a744-7b77-463b-a647-4f60a6f0a8da None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:43.607172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:43.608277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:43.609222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:43.609222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:43.609590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:43.610017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:43.612648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:43.630721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 917/1829] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:35:43.636492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0fc2f-bb26-4bde-bdab-b9a0c7bb2d67 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:43.650771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0fc2f-bb26-4bde-bdab-b9a0c7bb2d67 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:43.657884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:43.659468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:43.659745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:43.660009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:43.660853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:43.676685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:43.862356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 913/1830] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:43 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:43.868211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:43.883705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60dcf6-38f7-41d2-8ea2-89ca83212948 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:44.064071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:44.065498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:44.065658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:44.065913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:44.066567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:44.070970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:44.250595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 918/1831] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:43 2022] POST /identity/v3/users => generated 288 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:44.252534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:44.297190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b179751-3c20-4606-8b76-9a7870191fff admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:44.314241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:44.314241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:44.314241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:44.314241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:44.314848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:44.317049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:44.336572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 914/1832] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:44.341316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:44.367699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d273b24-0a06-433f-9f06-286fd5edc0a5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:44.396531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:44.397725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:44.398946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:44.398946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:44.400026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:44.405553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:44.417771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:44.420477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:44.451311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 919/1833] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:44.453437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:44.455666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f600d-c69b-406f-98f4-a3634f812832 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:44.469427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:44.470745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:44.471791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:44.473144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] PATH_INFO: `/v3/projects/c085b88bdbe34f2591f5f68e5c05c3f1` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:44.475964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c085b88bdbe34f2591f5f68e5c05c3f1)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:44.480640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:44.489848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771e5aa-8a90-4008-ab71-3b732785fed3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:44.501535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddae8a-bd6f-44cb-94fe-1c31480561d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:44.503132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddae8a-bd6f-44cb-94fe-1c31480561d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:44.505916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddae8a-bd6f-44cb-94fe-1c31480561d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:44.506591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddae8a-bd6f-44cb-94fe-1c31480561d7 admin admin] PATH_INFO: `/v3/projects/bf05d8aada7548378999f991a53760bf/users/372be3006c994bbc95b69ec97bc1df83/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:44.595371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ddae8a-bd6f-44cb-94fe-1c31480561d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf05d8aada7548378999f991a53760bf, user_id=372be3006c994bbc95b69ec97bc1df83, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:44.604311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ddae8a-bd6f-44cb-94fe-1c31480561d7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:44.797915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 920/1834] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:44 2022] PUT /identity/v3/projects/bf05d8aada7548378999f991a53760bf/users/372be3006c994bbc95b69ec97bc1df83/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:44.804897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c085b88bdbe34f2591f5f68e5c05c3f1', 'request_id': 'req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:44.820925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c085b88bdbe34f2591f5f68e5c05c3f1', 'request_id': 'req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:44.834993 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c085b88bdbe34f2591f5f68e5c05c3f1', 'request_id': 'req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:44.838367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9350de-8c73-4ae8-a878-eb017a0154cc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:44.838874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9350de-8c73-4ae8-a878-eb017a0154cc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:44.839376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9350de-8c73-4ae8-a878-eb017a0154cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:44.927152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1b9350de-8c73-4ae8-a878-eb017a0154cc None None] MFA Rules not processed for user `372be3006c994bbc95b69ec97bc1df83`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:44.928538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1b9350de-8c73-4ae8-a878-eb017a0154cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:45.042414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 921/1835] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:45.046702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9350de-8c73-4ae8-a878-eb017a0154cc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:45.077769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] The token cache is being invalidate because project c085b88bdbe34f2591f5f68e5c05c3f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:45.078617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:45.078823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9dcf2f98-ee2a-4456-a1ab-61cb0a8f7aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:45.091951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 915/1836] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:44 2022] DELETE /identity/v3/projects/c085b88bdbe34f2591f5f68e5c05c3f1 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:45.095123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913f12-7d7d-4195-a6c6-5693d54746f2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.096681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913f12-7d7d-4195-a6c6-5693d54746f2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.097456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913f12-7d7d-4195-a6c6-5693d54746f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.138101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-75913f12-7d7d-4195-a6c6-5693d54746f2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:45.142254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-75913f12-7d7d-4195-a6c6-5693d54746f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:45.160173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9350de-8c73-4ae8-a878-eb017a0154cc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:45.170968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:45.171630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.171947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.172226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.172884 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:45.174931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:45.239157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 916/1837] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:45.248411 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478404c-a1dd-4e41-b4ba-69fe06b75afa None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.248847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478404c-a1dd-4e41-b4ba-69fe06b75afa None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.249017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478404c-a1dd-4e41-b4ba-69fe06b75afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.270643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7478404c-a1dd-4e41-b4ba-69fe06b75afa None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:45.271801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7478404c-a1dd-4e41-b4ba-69fe06b75afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:45.282068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 917/1838] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:45.288990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478404c-a1dd-4e41-b4ba-69fe06b75afa None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:45.297563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478404c-a1dd-4e41-b4ba-69fe06b75afa None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:45.316037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:45.316682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.316963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.317234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.317898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:45.318295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:45.321498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:45.341507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 918/1839] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:35:45.344084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 922/1840] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:45 2022] POST /identity/v3/projects => generated 375 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:35:45.353852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:45.354363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:45.356369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:45.369140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3883887-f6da-4569-bf80-58970e4fe50d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:45.405309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:45.543739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:45.549152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.549152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.549152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.549152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:45.549152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:45.551672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d46875-fe9a-40a1-8523-51cfc362e607 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:45.561736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:45.562551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.562738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.562995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.563885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:45.565822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:45.734776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 919/1841] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:45 2022] POST /identity/v3/projects => generated 371 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:35:45.736214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 923/1842] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:45 2022] POST /identity/v3/users => generated 294 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:45.739479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:45.740608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:45.755729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766bb96-f2d5-46eb-af95-924c64edda27 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:45.849160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc31845-5ea5-4f43-b440-5517beb2917b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:45.862536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:45.863502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.863894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.864322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.865183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:45.872382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:45.875392 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:45.878528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.880551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.881346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.885405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:45.891100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:45.914101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 924/1843] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:45.919316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:45.933552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d814704-7e39-44f9-85b5-aad01d81b5a5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:45.939606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:45.940667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:45.941147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:45.941604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] PATH_INFO: `/v3/projects/e9118f3efcc74fb19ecafd2dafb0755e/users/0c231b47c94b4d79afe6aa0e6976e109/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:45.968009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9118f3efcc74fb19ecafd2dafb0755e, user_id=0c231b47c94b4d79afe6aa0e6976e109, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:45.972000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:46.062619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 920/1844] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:45 2022] POST /identity/v3/users => generated 286 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:35:46.069929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:46.090088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580f61a-e142-4d54-9920-3e1ba4e1662e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:46.104342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:46.104970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.105248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.105525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.106225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:46.107899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:46.124543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 921/1845] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:46.130981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:46.145534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785195f-b2fe-437b-aee1-6716c762473e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:46.155543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 925/1846] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:45 2022] PUT /identity/v3/projects/e9118f3efcc74fb19ecafd2dafb0755e/users/0c231b47c94b4d79afe6aa0e6976e109/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:46.161241 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:46.329277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:46.333964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.333964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.333964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.333964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:46.333964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:46.346037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 922/1847] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:46.347061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62412ab0-c631-42db-bbe6-ac4c99f267b1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:46.349441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:46.369969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:46.380089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.380599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.381220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.385079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fec16-c38e-43cc-b1e9-6c7683295631 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:46.385386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:46.389715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:46.401662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076d99-ce2a-463c-a359-5b8f449007d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:46.401662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076d99-ce2a-463c-a359-5b8f449007d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.402968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076d99-ce2a-463c-a359-5b8f449007d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.402968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076d99-ce2a-463c-a359-5b8f449007d9 admin admin] PATH_INFO: `/v3/projects/7ef593b74ea4401995a6f25ceab87f06/users/bb365399b0a745dd8d3be4bf6fe601bb/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.404125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 926/1848] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:35:46.408409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:46.422347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dc8a1-c47b-43a0-98fd-98763f68483f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:46.432400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e4f12-41c1-4fce-a431-24e33df2c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:46.433101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e4f12-41c1-4fce-a431-24e33df2c091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.433629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e4f12-41c1-4fce-a431-24e33df2c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.434030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e4f12-41c1-4fce-a431-24e33df2c091 admin admin] PATH_INFO: `/v3/projects/e9118f3efcc74fb19ecafd2dafb0755e/users/0c231b47c94b4d79afe6aa0e6976e109/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.437715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076d99-ce2a-463c-a359-5b8f449007d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef593b74ea4401995a6f25ceab87f06, user_id=bb365399b0a745dd8d3be4bf6fe601bb, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:46.437910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e4f12-41c1-4fce-a431-24e33df2c091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9118f3efcc74fb19ecafd2dafb0755e, user_id=0c231b47c94b4d79afe6aa0e6976e109, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:46.443040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076d99-ce2a-463c-a359-5b8f449007d9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:46.443207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e4f12-41c1-4fce-a431-24e33df2c091 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:46.454875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 927/1849] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:46 2022] PUT /identity/v3/projects/e9118f3efcc74fb19ecafd2dafb0755e/users/0c231b47c94b4d79afe6aa0e6976e109/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:46.467609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.468155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.468438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.502365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] MFA Rules not processed for user `0c231b47c94b4d79afe6aa0e6976e109`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:46.503685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:46.615232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 923/1850] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:46 2022] PUT /identity/v3/projects/7ef593b74ea4401995a6f25ceab87f06/users/bb365399b0a745dd8d3be4bf6fe601bb/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:46.628434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.628963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.629099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.642432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 928/1851] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:46.650246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:46.654973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:46.664178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] MFA Rules not processed for user `bb365399b0a745dd8d3be4bf6fe601bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:46.665446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:46.758800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1401b9b-7388-4de5-9a30-b7755abbd049 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:46.764542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 924/1852] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:46.852375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:46.853025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:46.853324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:46.853572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:46.855685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:46.933672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:46.935080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:46.946048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 929/1853] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:35:47.455386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:48.008550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:48.011150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:48.032272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27dc87-8cb9-4c37-ab85-a6337cdef4fb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:48.038260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f7198-0d4a-43ed-ada5-2c4b5a9420d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:48.039250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7198-0d4a-43ed-ada5-2c4b5a9420d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:48.039809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7198-0d4a-43ed-ada5-2c4b5a9420d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:48.040049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7198-0d4a-43ed-ada5-2c4b5a9420d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:48.051126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7198-0d4a-43ed-ada5-2c4b5a9420d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:48.052947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7198-0d4a-43ed-ada5-2c4b5a9420d2 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:48.065065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 925/1854] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:48.096998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:48.267518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f06a1-85e6-4237-b0d5-7f0752c4bdab service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:48.291330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:48.292095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:48.292385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:48.292638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] PATH_INFO: `/v3/users/bb365399b0a745dd8d3be4bf6fe601bb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:48.294346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb365399b0a745dd8d3be4bf6fe601bb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:48.296285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:48.435726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:48.666385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:48.692363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb365399b0a745dd8d3be4bf6fe601bb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:48.693584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb365399b0a745dd8d3be4bf6fe601bb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:48.710336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb365399b0a745dd8d3be4bf6fe601bb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:48.711430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb365399b0a745dd8d3be4bf6fe601bb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:48.819388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb365399b0a745dd8d3be4bf6fe601bb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:48.867991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 930/1855] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:48 2022] DELETE /identity/v3/users/bb365399b0a745dd8d3be4bf6fe601bb => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:48.876405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:48.877648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:48.879438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:48.922074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:48.924151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:49.037110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 926/1856] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:49.064537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:49.066351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:49.150257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56794ca-def0-42c5-b5d1-cfac59bf350e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:49.167122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fea08-3152-410b-b97c-1245bcd8cf27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:49.167766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fea08-3152-410b-b97c-1245bcd8cf27 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:49.168055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fea08-3152-410b-b97c-1245bcd8cf27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:49.168325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fea08-3152-410b-b97c-1245bcd8cf27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:49.181004 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fea08-3152-410b-b97c-1245bcd8cf27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:49.182706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fea08-3152-410b-b97c-1245bcd8cf27 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:49.210743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 931/1857] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:50.072338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:50.073814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:50.089004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f193b3-68bd-4a3e-840d-64a0e95210e3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:50.099785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:50.101366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:50.101841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:50.102339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] PATH_INFO: `/v3/projects/7ef593b74ea4401995a6f25ceab87f06` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:50.105959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef593b74ea4401995a6f25ceab87f06)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:50.110319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:50.283492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef593b74ea4401995a6f25ceab87f06', 'request_id': 'req-d63ca159-69c9-4b45-bae4-cc96536afb3f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:50.290110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef593b74ea4401995a6f25ceab87f06', 'request_id': 'req-d63ca159-69c9-4b45-bae4-cc96536afb3f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:50.297598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef593b74ea4401995a6f25ceab87f06', 'request_id': 'req-d63ca159-69c9-4b45-bae4-cc96536afb3f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:50.459061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] The token cache is being invalidate because project 7ef593b74ea4401995a6f25ceab87f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:50.460869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:50.462152 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d63ca159-69c9-4b45-bae4-cc96536afb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:50.464899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 927/1858] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:50 2022] DELETE /identity/v3/projects/7ef593b74ea4401995a6f25ceab87f06 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:50.471661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52313c-5ce4-41d3-9334-d0fb6ede228b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:50.472391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52313c-5ce4-41d3-9334-d0fb6ede228b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:50.472391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52313c-5ce4-41d3-9334-d0fb6ede228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:50.514731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0e52313c-5ce4-41d3-9334-d0fb6ede228b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:50.516084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0e52313c-5ce4-41d3-9334-d0fb6ede228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:50.634753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 932/1859] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:50.643264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c129-5dc0-466b-a1fb-355266c559ea None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:50.643745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c129-5dc0-466b-a1fb-355266c559ea None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:50.644170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c129-5dc0-466b-a1fb-355266c559ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:50.669838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4ce0c129-5dc0-466b-a1fb-355266c559ea None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:50.671524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4ce0c129-5dc0-466b-a1fb-355266c559ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:50.701159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 928/1860] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:50.707874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52313c-5ce4-41d3-9334-d0fb6ede228b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:50.745824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52313c-5ce4-41d3-9334-d0fb6ede228b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:50.762266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:50.763002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:50.763173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:50.763534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:50.764098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:50.764407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:50.769402 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:50.810255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0c129-5dc0-466b-a1fb-355266c559ea None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:50.823796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 933/1861] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:35:50.824646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0c129-5dc0-466b-a1fb-355266c559ea None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:50.840274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:50.841242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:50.841520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:50.841803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:50.842490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:50.844452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:50.947246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:50.950877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:50.972663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ee6fd-c5e2-4391-9f63-0b515acf3b8c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:50.980334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:50.981177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:50.989197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:50.991367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] PATH_INFO: `/v3/users/372be3006c994bbc95b69ec97bc1df83` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:50.995512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=372be3006c994bbc95b69ec97bc1df83)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:51.010496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:51.025811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:51.026363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 929/1862] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:50 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:35:51.032183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:51.158993 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dbd6d-f38f-4feb-a6ea-7ed2d0d240dd admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:51.173774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:51.174981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:51.175547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:51.176125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:51.177353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:51.181854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:51.338323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 930/1863] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:51 2022] POST /identity/v3/users => generated 287 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:51.344654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:51.376219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d264d-9d22-4cb1-87bc-bb933fd388ce admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:51.391463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:51.392109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372be3006c994bbc95b69ec97bc1df83'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:51.393964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:51.394660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:51.395213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372be3006c994bbc95b69ec97bc1df83'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:51.395875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:51.398865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:51.401023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:51.410441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372be3006c994bbc95b69ec97bc1df83'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:51.411235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372be3006c994bbc95b69ec97bc1df83'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:51.421454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 931/1864] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:51.422735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:51.424998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:51.440797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4846-495a-4875-827a-2ebdec7cd3e2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:51.446739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:51.447380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:51.447661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:51.447936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:51.448603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:51.450490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:51.466688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 932/1865] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:51.471227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:51.483215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a8ca0-a4d4-41be-88fb-5ab5d8079b31 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:51.488686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc26a3-87b2-48c7-ae96-cce68bf15eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:51.489474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc26a3-87b2-48c7-ae96-cce68bf15eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:51.489821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc26a3-87b2-48c7-ae96-cce68bf15eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:51.490205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc26a3-87b2-48c7-ae96-cce68bf15eb8 admin admin] PATH_INFO: `/v3/projects/079e4fab64524133be9ecb0548c63dcb/users/ed921f9500944e19a206b7605ee33b20/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:51.509680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1e43d109-b2ff-4a9b-a52c-62f499bd27dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372be3006c994bbc95b69ec97bc1df83'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:51.515495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc26a3-87b2-48c7-ae96-cce68bf15eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079e4fab64524133be9ecb0548c63dcb, user_id=ed921f9500944e19a206b7605ee33b20, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:51.528860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc26a3-87b2-48c7-ae96-cce68bf15eb8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:51.543777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 934/1866] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:50 2022] DELETE /identity/v3/users/372be3006c994bbc95b69ec97bc1df83 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:51.550749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:51.551347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:51.551781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:51.580988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:51.582944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:51.656733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 933/1867] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:51 2022] PUT /identity/v3/projects/079e4fab64524133be9ecb0548c63dcb/users/ed921f9500944e19a206b7605ee33b20/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:51.660302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:51.661400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:51.662065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:51.698908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 935/1868] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:51.727329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:51.730182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:51.734297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] MFA Rules not processed for user `ed921f9500944e19a206b7605ee33b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:51.738443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:51.812303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bc3d4-67d2-41a6-9b4a-ebd861af31a2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:51.864606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 934/1869] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:51.890383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:51.891997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:51.961642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:51.962939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:51.963496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:51.965867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:52.039624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7558f-c434-403c-bbf2-0e8beae730ae None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:52.048185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:52.048991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:52.049562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:52.050456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:52.072180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:52.074949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:52.093582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 935/1870] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:52.128907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:52.133031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:52.155630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 936/1871] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:35:53.397522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:53.686090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:53.688398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:53.708534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363050f-ff50-4e4d-9170-d0bc006c5d7d service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:53.728532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:53.729555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:53.729985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:53.730537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] PATH_INFO: `/v3/projects/bf05d8aada7548378999f991a53760bf` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:53.732824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf05d8aada7548378999f991a53760bf)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:53.739536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:53.919651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf05d8aada7548378999f991a53760bf', 'request_id': 'req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:53.932564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf05d8aada7548378999f991a53760bf', 'request_id': 'req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:53.946981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf05d8aada7548378999f991a53760bf', 'request_id': 'req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.066494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] The token cache is being invalidate because project bf05d8aada7548378999f991a53760bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:54.066877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.068066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.071036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 936/1872] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:53 2022] DELETE /identity/v3/projects/bf05d8aada7548378999f991a53760bf => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:54.075792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:54.089564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee1bec-bdb3-4be2-a9b2-75dc1a333b21 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:54.176697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:54.177446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:54.177789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:54.178171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] PATH_INFO: `/v3/users/0c231b47c94b4d79afe6aa0e6976e109` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:54.179856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c231b47c94b4d79afe6aa0e6976e109)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:54.181881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:54.549489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c231b47c94b4d79afe6aa0e6976e109'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.550640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c231b47c94b4d79afe6aa0e6976e109'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.565219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c231b47c94b4d79afe6aa0e6976e109'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.566383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c231b47c94b4d79afe6aa0e6976e109'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.721793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fc2491fe-3d73-4cd5-aae9-68397f6cc116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c231b47c94b4d79afe6aa0e6976e109'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:54.801845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 937/1873] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:35:54 2022] DELETE /identity/v3/users/0c231b47c94b4d79afe6aa0e6976e109 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:55.114762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:55.722932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:35:56.094814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:56.096578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:56.157664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc055e2a-be93-4c76-9a19-ca6e18e9c4dc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:56.375676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:56.377610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:56.378378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:56.379206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] PATH_INFO: `/v3/projects/e9118f3efcc74fb19ecafd2dafb0755e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:56.388630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9118f3efcc74fb19ecafd2dafb0755e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:56.393838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:56.631049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9118f3efcc74fb19ecafd2dafb0755e', 'request_id': 'req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:56.643081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9118f3efcc74fb19ecafd2dafb0755e', 'request_id': 'req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:56.657364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9118f3efcc74fb19ecafd2dafb0755e', 'request_id': 'req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:56.784145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] The token cache is being invalidate because project e9118f3efcc74fb19ecafd2dafb0755e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:35:56.784731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:56.786057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0d9bcde9-09f6-4d73-a2dc-9b36a291da2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:35:56.792957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 937/1874] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:35:56 2022] DELETE /identity/v3/projects/e9118f3efcc74fb19ecafd2dafb0755e => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:35:56.795534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9a0f-c144-4192-950d-498662526d9e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:56.796902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9a0f-c144-4192-950d-498662526d9e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:56.797343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9a0f-c144-4192-950d-498662526d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:56.837329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f76b9a0f-c144-4192-950d-498662526d9e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:56.839533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f76b9a0f-c144-4192-950d-498662526d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:56.974235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 938/1875] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:35:56.987797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f3065-2834-44e1-ad26-788036ab3dd4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:56.988783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f3065-2834-44e1-ad26-788036ab3dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:56.999906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f3065-2834-44e1-ad26-788036ab3dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:57.055025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d30f3065-2834-44e1-ad26-788036ab3dd4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:57.056712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d30f3065-2834-44e1-ad26-788036ab3dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:57.077520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 938/1876] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:57.080065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9a0f-c144-4192-950d-498662526d9e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:57.097129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9a0f-c144-4192-950d-498662526d9e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:57.105917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:57.107170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:57.108306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:57.108502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:57.109855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:35:57.110423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:57.113437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:57.148863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 939/1877] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:35:57.150192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f3065-2834-44e1-ad26-788036ab3dd4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:57.161730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f3065-2834-44e1-ad26-788036ab3dd4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:57.169350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:57.170391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:57.173227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:57.173227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:57.173227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:57.173684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:57.328775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 939/1878] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:35:57 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:57.330106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:57.385587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970d9a6-b0d9-4147-aa9e-972151aa2a9a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:57.533248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:57.534683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:57.534859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:57.535125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:57.535790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:57.541557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:57.671986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 940/1879] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:35:57 2022] POST /identity/v3/users => generated 286 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:35:57.679179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:57.736711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c1f98-bd83-4163-a131-b7725c22935a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:57.747538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:57.748542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:57.748990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:57.749323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:57.750581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:57.752483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:57.778357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 940/1880] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:57.779444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:57.794413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7125fd2-6645-46ce-a1c3-433c16f6ee32 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:57.802667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1257f-c9b4-4d5f-b7d2-97d6d9847f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:57.804637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1257f-c9b4-4d5f-b7d2-97d6d9847f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:57.804867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1257f-c9b4-4d5f-b7d2-97d6d9847f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:57.805649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1257f-c9b4-4d5f-b7d2-97d6d9847f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:57.807077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1257f-c9b4-4d5f-b7d2-97d6d9847f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:57.810401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1257f-c9b4-4d5f-b7d2-97d6d9847f2e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:57.832506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 941/1881] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:35:57.838312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:57.853150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05fd9a-abc8-4f19-9d80-37903778951f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:57.858474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:57.859177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:57.859458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:57.859932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] PATH_INFO: `/v3/projects/aa7908a26d224db1bf2e074c5f25b315/users/6e729065f1524fd9bf9ef3c6b42f7a5e/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:57.879673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7908a26d224db1bf2e074c5f25b315, user_id=6e729065f1524fd9bf9ef3c6b42f7a5e, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:57.883122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:58.068432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 941/1882] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:35:57 2022] PUT /identity/v3/projects/aa7908a26d224db1bf2e074c5f25b315/users/6e729065f1524fd9bf9ef3c6b42f7a5e/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:35:58.074463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2c72-4944-4e9f-b9ee-8f93039c7987 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:58.074860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2c72-4944-4e9f-b9ee-8f93039c7987 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:58.075244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2c72-4944-4e9f-b9ee-8f93039c7987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:58.124449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9dac2c72-4944-4e9f-b9ee-8f93039c7987 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:35:58.126917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9dac2c72-4944-4e9f-b9ee-8f93039c7987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:58.237497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 942/1883] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:35:58.254823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:35:58.261011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:35:58.369062 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744b113-8c27-46b2-8757-a4eed8ed8f8c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:35:58.376563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:35:58.377975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:35:58.382762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:35:58.382762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:35:58.397262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:35:58.399616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:35:58.423274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 942/1884] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:35:59.614630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:00.141487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:00.644945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a58c4-7d56-4137-ab1d-88a77c12e2f2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:00.645790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a58c4-7d56-4137-ab1d-88a77c12e2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:00.646518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a58c4-7d56-4137-ab1d-88a77c12e2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:00.701302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-615a58c4-7d56-4137-ab1d-88a77c12e2f2 None None] MFA Rules not processed for user `6e729065f1524fd9bf9ef3c6b42f7a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:00.703119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-615a58c4-7d56-4137-ab1d-88a77c12e2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:00.793723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 943/1885] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:00.806278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:00.808843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:00.911320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237f37-bf2f-4361-877c-707ed52404cd service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:00.916154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:00.916790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:00.917103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:00.917378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:00.929047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:00.930607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:00.942193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 943/1886] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:36:02.201570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:03.143973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:03.526804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:03.732622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:04.411234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:04.493411 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:05.322226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:05.393550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:06.213632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:06.237527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085d66-dbfa-4494-becc-5c5911bc666b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:06.237527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085d66-dbfa-4494-becc-5c5911bc666b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:06.237527 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf085d66-dbfa-4494-becc-5c5911bc666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:06.262609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cf085d66-dbfa-4494-becc-5c5911bc666b None None] MFA Rules not processed for user `6e729065f1524fd9bf9ef3c6b42f7a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:06.264063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cf085d66-dbfa-4494-becc-5c5911bc666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:06.273448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 944/1887] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:08.688443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:09.632024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:10.528220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:11.272555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:11.272555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:11.296068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e9e26-f544-46da-9bce-c5ea5601aa03 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:11.305114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:11.306266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:11.312007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:11.312007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] PATH_INFO: `/v3/users/ed921f9500944e19a206b7605ee33b20` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:11.312007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed921f9500944e19a206b7605ee33b20)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:11.313407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:11.674701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed921f9500944e19a206b7605ee33b20'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:11.675614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed921f9500944e19a206b7605ee33b20'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:11.676382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed921f9500944e19a206b7605ee33b20'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:11.678682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:11.806390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed921f9500944e19a206b7605ee33b20'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:11.817763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed921f9500944e19a206b7605ee33b20'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:11.865466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 944/1888] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:36:11 2022] DELETE /identity/v3/users/ed921f9500944e19a206b7605ee33b20 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:11.873120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:11.873875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:11.874315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:11.922101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:11.923423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:12.017542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 945/1889] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:12.029425 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:12.031100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:12.101911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7257304-d5ad-4f3b-8fdf-91a9b181715b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:12.106013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d97c8-1a6c-4f41-a3a4-0b5092368baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:12.106675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d97c8-1a6c-4f41-a3a4-0b5092368baf service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:12.106969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d97c8-1a6c-4f41-a3a4-0b5092368baf service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:12.107231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d97c8-1a6c-4f41-a3a4-0b5092368baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:12.120596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d97c8-1a6c-4f41-a3a4-0b5092368baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:12.130854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d97c8-1a6c-4f41-a3a4-0b5092368baf service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:12.141049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 945/1890] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:36:13.053538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:13.055958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:13.079484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb992f-7b01-4473-a52f-dd6b91ae3370 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:13.092454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:13.093920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:13.096347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:13.097608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] PATH_INFO: `/v3/projects/079e4fab64524133be9ecb0548c63dcb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:13.099668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079e4fab64524133be9ecb0548c63dcb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:13.103412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:13.256457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079e4fab64524133be9ecb0548c63dcb', 'request_id': 'req-72244087-19ba-4b0e-8042-3c5e1c0b8f16'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:13.267145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079e4fab64524133be9ecb0548c63dcb', 'request_id': 'req-72244087-19ba-4b0e-8042-3c5e1c0b8f16'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:13.406508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079e4fab64524133be9ecb0548c63dcb', 'request_id': 'req-72244087-19ba-4b0e-8042-3c5e1c0b8f16'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:13.426755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] The token cache is being invalidate because project 079e4fab64524133be9ecb0548c63dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:13.427271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:13.429812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-72244087-19ba-4b0e-8042-3c5e1c0b8f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:13.432494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 946/1891] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:13 2022] DELETE /identity/v3/projects/079e4fab64524133be9ecb0548c63dcb => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:13.440812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696614f-aed6-4d96-ba18-4267cbe7eb67 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:13.441368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696614f-aed6-4d96-ba18-4267cbe7eb67 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:13.441681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696614f-aed6-4d96-ba18-4267cbe7eb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:13.473833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b696614f-aed6-4d96-ba18-4267cbe7eb67 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:13.475726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b696614f-aed6-4d96-ba18-4267cbe7eb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:13.563440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 946/1892] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:13.572791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d1bc-6aab-4e50-b59f-644f0d6f3603 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:13.573084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d1bc-6aab-4e50-b59f-644f0d6f3603 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:13.573367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d1bc-6aab-4e50-b59f-644f0d6f3603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:13.596586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0136d1bc-6aab-4e50-b59f-644f0d6f3603 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:13.598317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0136d1bc-6aab-4e50-b59f-644f0d6f3603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:13.607556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 947/1893] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:13.612286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696614f-aed6-4d96-ba18-4267cbe7eb67 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:13.624120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696614f-aed6-4d96-ba18-4267cbe7eb67 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:13.629226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:13.629859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:13.630183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:13.630461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:13.631698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:36:13.632015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:13.633714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:13.648061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 947/1894] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:36:13.652800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d1bc-6aab-4e50-b59f-644f0d6f3603 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:13.663861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d1bc-6aab-4e50-b59f-644f0d6f3603 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:13.669273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:13.669929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:13.670291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:13.670561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:13.671232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:13.673253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:13.828076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 948/1895] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:13 2022] POST /identity/v3/projects => generated 397 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:13.833382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:13.854086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990eba-2e10-4b0d-88dd-ad9bf21d71f4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:13.985044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:14.014468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:14.017045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:14.022767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:14.022767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:14.022767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:14.026735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:14.165057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 948/1896] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:13 2022] POST /identity/v3/users => generated 299 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:14.168146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:14.183817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93d4be-6986-4028-8da3-4e3893912ac4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:14.198049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:14.211139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:14.211433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:14.214213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:14.217696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:14.220319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:14.234731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 949/1897] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:14.239351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:14.253265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b9b1-df36-44c3-aaac-3c22f536b39d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:14.262075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a329af-d89f-4af0-a40a-06db643b52d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:14.263148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a329af-d89f-4af0-a40a-06db643b52d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:14.263651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a329af-d89f-4af0-a40a-06db643b52d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:14.264145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a329af-d89f-4af0-a40a-06db643b52d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:14.265370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a329af-d89f-4af0-a40a-06db643b52d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:14.268623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a329af-d89f-4af0-a40a-06db643b52d9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:14.297373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 949/1898] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:14.299517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:14.316123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9899-d260-48fc-9c0a-76784c24c739 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:14.323616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:14.324741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:14.325199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:14.325635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] PATH_INFO: `/v3/projects/88a7d93a357a49e6b0ebf69acf41eb42/users/9a9a3e8b356f48f3b0a6e300b4971e73/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:14.351614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a7d93a357a49e6b0ebf69acf41eb42, user_id=9a9a3e8b356f48f3b0a6e300b4971e73, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:14.354676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:14.443596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:14.465316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 950/1899] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:14 2022] PUT /identity/v3/projects/88a7d93a357a49e6b0ebf69acf41eb42/users/9a9a3e8b356f48f3b0a6e300b4971e73/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:14.467487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efb8c1-e689-4262-b1a6-03f5f4a2fb89 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:14.468277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efb8c1-e689-4262-b1a6-03f5f4a2fb89 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:14.468277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efb8c1-e689-4262-b1a6-03f5f4a2fb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:14.510133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e8efb8c1-e689-4262-b1a6-03f5f4a2fb89 None None] MFA Rules not processed for user `9a9a3e8b356f48f3b0a6e300b4971e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:14.511436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e8efb8c1-e689-4262-b1a6-03f5f4a2fb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:14.612616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 950/1900] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:14 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:36:14.632835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:14.635993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:14.726668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93df950-a4c2-4cca-b28b-44b509678076 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:14.730799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ad260-affe-4caf-a62e-4881e266fef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:14.731431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ad260-affe-4caf-a62e-4881e266fef4 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:14.731707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ad260-affe-4caf-a62e-4881e266fef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:14.731976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ad260-affe-4caf-a62e-4881e266fef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:14.741746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ad260-affe-4caf-a62e-4881e266fef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:14.743230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ad260-affe-4caf-a62e-4881e266fef4 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:14.754988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 951/1901] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:14 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:36:24.607103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:25.171693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=26 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:27.020237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efb8c1-e689-4262-b1a6-03f5f4a2fb89 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:27.181094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efb8c1-e689-4262-b1a6-03f5f4a2fb89 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:27.189925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:27.191483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:27.191986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:27.192430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] PATH_INFO: `/v3/users/9a9a3e8b356f48f3b0a6e300b4971e73` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:27.195704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a9a3e8b356f48f3b0a6e300b4971e73)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:27.200008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:27.508096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a9a3e8b356f48f3b0a6e300b4971e73'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:27.510201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a9a3e8b356f48f3b0a6e300b4971e73'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:27.511265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a9a3e8b356f48f3b0a6e300b4971e73'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:27.610753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a9a3e8b356f48f3b0a6e300b4971e73'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:27.628420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a9a3e8b356f48f3b0a6e300b4971e73'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:27.691433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 951/1902] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:36:27 2022] DELETE /identity/v3/users/9a9a3e8b356f48f3b0a6e300b4971e73 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:27.698333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:27.699081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:27.699232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:27.735695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:27.738694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:27.918313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 952/1903] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:27.936971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:27.939856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:28.074511 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9865f29-9ccc-4f0e-8e20-170286ff301d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:28.079404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff0269-7aee-456c-bfca-661a43ef67c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:28.080825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff0269-7aee-456c-bfca-661a43ef67c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:28.081285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff0269-7aee-456c-bfca-661a43ef67c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:28.081726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff0269-7aee-456c-bfca-661a43ef67c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:28.097280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff0269-7aee-456c-bfca-661a43ef67c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:28.099218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff0269-7aee-456c-bfca-661a43ef67c6 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:28.120302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 952/1904] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:36:29.011949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:29.013816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:29.029056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641f6f5-177c-4f4e-b293-3ca60d9a1358 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:29.034485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:29.035403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:29.035693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:29.035954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] PATH_INFO: `/v3/projects/88a7d93a357a49e6b0ebf69acf41eb42` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:29.037269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a7d93a357a49e6b0ebf69acf41eb42)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:29.039332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:29.216271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a7d93a357a49e6b0ebf69acf41eb42', 'request_id': 'req-4a8bddf9-3143-4661-9731-5fbc25178e84'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:29.225399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a7d93a357a49e6b0ebf69acf41eb42', 'request_id': 'req-4a8bddf9-3143-4661-9731-5fbc25178e84'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:29.349274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a7d93a357a49e6b0ebf69acf41eb42', 'request_id': 'req-4a8bddf9-3143-4661-9731-5fbc25178e84'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:29.380869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] The token cache is being invalidate because project 88a7d93a357a49e6b0ebf69acf41eb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:29.381382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:29.382558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4a8bddf9-3143-4661-9731-5fbc25178e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:29.385583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 953/1905] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:29 2022] DELETE /identity/v3/projects/88a7d93a357a49e6b0ebf69acf41eb42 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:29.391922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3ce9c-1516-4c67-90d5-90f239d83f53 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:29.392490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3ce9c-1516-4c67-90d5-90f239d83f53 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:29.392923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3ce9c-1516-4c67-90d5-90f239d83f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:29.430196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-cee3ce9c-1516-4c67-90d5-90f239d83f53 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:29.433025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cee3ce9c-1516-4c67-90d5-90f239d83f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:29.468537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:29.547504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 953/1906] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:29.554443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010b0a4-d995-4022-988b-986b39ca2fa9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:29.555597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010b0a4-d995-4022-988b-986b39ca2fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:29.556173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010b0a4-d995-4022-988b-986b39ca2fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:29.584333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e010b0a4-d995-4022-988b-986b39ca2fa9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:29.585691 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e010b0a4-d995-4022-988b-986b39ca2fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:29.595896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 954/1907] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:29.601231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3ce9c-1516-4c67-90d5-90f239d83f53 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:29.618225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3ce9c-1516-4c67-90d5-90f239d83f53 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:29.637083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:29.637083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:29.637083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:29.637083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:29.638161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:36:29.638161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:29.638161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:29.653322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 954/1908] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:36:29.659206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010b0a4-d995-4022-988b-986b39ca2fa9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:29.678964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010b0a4-d995-4022-988b-986b39ca2fa9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:29.685007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:29.686121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:29.686420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:29.686782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:29.687581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:29.691954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:29.843269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 955/1909] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:29 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:29.845287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:29.868365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860577c9-6f90-4932-bd10-8d0872a959ce admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:29.950250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:29.951107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:29.951509 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:29.952059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:29.953829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:29.957741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:30.140110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 955/1910] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:29 2022] POST /identity/v3/users => generated 292 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:30.146088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:30.173335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df7735-36af-4396-b6e1-a8469bd29422 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:30.180595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:30.181537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:30.181926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:30.182469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:30.183286 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:30.185359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:30.198399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 956/1911] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:30.204259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:30.216139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65769a0-70bd-416c-b900-9bdef01600fc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:30.221868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7900-85b4-458e-8cc7-dd9cc5178cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:30.222986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7900-85b4-458e-8cc7-dd9cc5178cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:30.223348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7900-85b4-458e-8cc7-dd9cc5178cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:30.223348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7900-85b4-458e-8cc7-dd9cc5178cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:30.224021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7900-85b4-458e-8cc7-dd9cc5178cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:30.226500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7900-85b4-458e-8cc7-dd9cc5178cbd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:30.242977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 956/1912] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:30.249816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:30.262210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a0863-1b78-490e-b01a-31cfe0ca27a9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:30.267657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:30.268354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:30.268701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:30.268983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] PATH_INFO: `/v3/projects/853f7f0f1616478c9744f93293be553e/users/de720e2d8a0f4b5f898e00ce90befd58/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:30.292275 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=853f7f0f1616478c9744f93293be553e, user_id=de720e2d8a0f4b5f898e00ce90befd58, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:30.295155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:30.437000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 957/1913] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:30 2022] PUT /identity/v3/projects/853f7f0f1616478c9744f93293be553e/users/de720e2d8a0f4b5f898e00ce90befd58/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:30.442541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:30.443090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:30.443760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:30.466870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:30.468261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:30.539305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 957/1914] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:30.550209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:30.552269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:30.577512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:30.620563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e19f33-e7c2-4290-a378-602e56aad7a7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:30.624732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189e38e-3540-41c9-a217-dcbef287f41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:30.625366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189e38e-3540-41c9-a217-dcbef287f41d service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:30.625650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189e38e-3540-41c9-a217-dcbef287f41d service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:30.625920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189e38e-3540-41c9-a217-dcbef287f41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:30.637530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189e38e-3540-41c9-a217-dcbef287f41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:30.639343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189e38e-3540-41c9-a217-dcbef287f41d service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:30.656084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 958/1915] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:36:31.741206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:31.744635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:31.770825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a5971-3483-47cc-80dd-a68196c013e2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:31.781360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:31.782700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:31.783277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:31.783923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] PATH_INFO: `/v3/users/7084143e092045758a231ce4298e52f1` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:31.786315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7084143e092045758a231ce4298e52f1)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:31.791285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:31.829467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:32.149572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7084143e092045758a231ce4298e52f1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:32.150499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7084143e092045758a231ce4298e52f1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:32.151250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7084143e092045758a231ce4298e52f1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:32.252706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7084143e092045758a231ce4298e52f1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:32.275474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7084143e092045758a231ce4298e52f1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:32.322840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 958/1916] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:36:31 2022] DELETE /identity/v3/users/7084143e092045758a231ce4298e52f1 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:32.329640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:32.330524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:32.330749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:32.359129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:32.363250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:32.451285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 959/1917] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:32.462616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:32.465182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:32.583212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12d166-0b44-4137-8d79-4467f0d57b28 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:32.591526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:32.593542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:32.593542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:32.593542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:32.614688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:32.616315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:32.630354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 959/1918] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:36:34.237178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:34.239770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:34.270897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28801d5d-127c-4228-8f96-8a0971cbdc86 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:34.311174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:34.312204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:34.312676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:34.313166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] PATH_INFO: `/v3/projects/e7af817492f44ad9b80060db5a5e3ab7` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:34.314980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7af817492f44ad9b80060db5a5e3ab7)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:34.317185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:34.525174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7af817492f44ad9b80060db5a5e3ab7', 'request_id': 'req-0771ddda-1809-4caf-aa0c-115587486046'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:34.553867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7af817492f44ad9b80060db5a5e3ab7', 'request_id': 'req-0771ddda-1809-4caf-aa0c-115587486046'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:34.669381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7af817492f44ad9b80060db5a5e3ab7', 'request_id': 'req-0771ddda-1809-4caf-aa0c-115587486046'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:34.683833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] The token cache is being invalidate because project e7af817492f44ad9b80060db5a5e3ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:34.684135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:34.685026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:34.686955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 960/1919] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:34 2022] DELETE /identity/v3/projects/e7af817492f44ad9b80060db5a5e3ab7 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:34.691609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:34.705685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b948-3dd7-4692-8b14-70010a2b9f5e service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:34.789604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:34.791016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:34.791523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:34.792028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] PATH_INFO: `/v3/users/c69112c520d3477a87f730832c5baeb8` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:34.793980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c69112c520d3477a87f730832c5baeb8)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:34.797260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:35.281130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:35.285756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c69112c520d3477a87f730832c5baeb8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:35.289519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c69112c520d3477a87f730832c5baeb8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:35.291999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c69112c520d3477a87f730832c5baeb8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:35.407406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c69112c520d3477a87f730832c5baeb8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:35.422973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b72797d6-b2d9-452f-9f44-fd0d02feffe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c69112c520d3477a87f730832c5baeb8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:35.447455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 960/1920] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:36:34 2022] DELETE /identity/v3/users/c69112c520d3477a87f730832c5baeb8 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:36.874564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:36.874564 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:36.918390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ddda-1809-4caf-aa0c-115587486046 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:37.007905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:37.040819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:37.041834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:37.042362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:37.043094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] PATH_INFO: `/v3/projects/bc85bbed5d324a298b03a3cdab99c40c` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:37.045184 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc85bbed5d324a298b03a3cdab99c40c)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:37.051823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:37.321437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc85bbed5d324a298b03a3cdab99c40c', 'request_id': 'req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:37.355087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc85bbed5d324a298b03a3cdab99c40c', 'request_id': 'req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:37.469117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc85bbed5d324a298b03a3cdab99c40c', 'request_id': 'req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:37.493915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] The token cache is being invalidate because project bc85bbed5d324a298b03a3cdab99c40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:37.494299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:37.494981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fcbdb471-56fa-42ff-b249-a0f73fd7c15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:37.496715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 961/1921] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:36 2022] DELETE /identity/v3/projects/bc85bbed5d324a298b03a3cdab99c40c => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:37.510432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1a9a2-b2cc-4dc2-85e8-39c52553e937 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:37.512424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1a9a2-b2cc-4dc2-85e8-39c52553e937 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:37.516093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1a9a2-b2cc-4dc2-85e8-39c52553e937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:37.593773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-dbe1a9a2-b2cc-4dc2-85e8-39c52553e937 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:37.599361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dbe1a9a2-b2cc-4dc2-85e8-39c52553e937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:37.734299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 961/1922] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:37.737795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26bb62-59b4-4ba9-90ed-078e0e8be4da None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:37.738503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26bb62-59b4-4ba9-90ed-078e0e8be4da None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:37.739317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26bb62-59b4-4ba9-90ed-078e0e8be4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:37.779534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-af26bb62-59b4-4ba9-90ed-078e0e8be4da None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:37.779876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-af26bb62-59b4-4ba9-90ed-078e0e8be4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:37.793331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 962/1923] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:37.794277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1a9a2-b2cc-4dc2-85e8-39c52553e937 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:37.822132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1a9a2-b2cc-4dc2-85e8-39c52553e937 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:37.827565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:37.828372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:37.828723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:37.829061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:37.829800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:36:37.830205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:37.832076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:37.851854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 962/1924] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:36:37.853939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26bb62-59b4-4ba9-90ed-078e0e8be4da None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:37.874199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26bb62-59b4-4ba9-90ed-078e0e8be4da None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:37.885673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:37.886801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:37.887317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:37.887815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:37.889017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:37.893207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:38.081891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 963/1925] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:37 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:38.088601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:38.103945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c3ec-9473-4b42-86fb-8a1ff3d1f718 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:38.225115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:38.226960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:38.227429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:38.227890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:38.228843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:38.232608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:38.356785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 963/1926] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:38 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:38.367681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:38.407007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431dfc8-1a59-4a5d-9575-1e1c2520f13d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:38.418683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:38.419942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:38.420574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:38.421407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:38.422844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:38.426970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:38.467246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 964/1927] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:38.473310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:38.493359 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6a742-f471-4d37-83ba-84a95d0d27a6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:38.502936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5ec2f-ba36-482b-9059-8b53fe0d3966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:38.503523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ec2f-ba36-482b-9059-8b53fe0d3966 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:38.504023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ec2f-ba36-482b-9059-8b53fe0d3966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:38.504508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ec2f-ba36-482b-9059-8b53fe0d3966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:38.505840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5ec2f-ba36-482b-9059-8b53fe0d3966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:38.509670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5ec2f-ba36-482b-9059-8b53fe0d3966 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:38.552266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 964/1928] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:38.556754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:38.587003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51180fd-17cc-4285-b615-4cfd5bc5d62d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:38.595326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:38.596030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:38.596304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:38.596577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] PATH_INFO: `/v3/projects/66a50940607144d49d9036cddba73f11/users/b908d5c7e7244b088fb177d2dec2731d/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:38.634535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a50940607144d49d9036cddba73f11, user_id=b908d5c7e7244b088fb177d2dec2731d, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:38.637486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:38.779937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 965/1929] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:38 2022] PUT /identity/v3/projects/66a50940607144d49d9036cddba73f11/users/b908d5c7e7244b088fb177d2dec2731d/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:38.806086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817030d7-d40a-45f0-bebf-87bb1d500686 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:38.806949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817030d7-d40a-45f0-bebf-87bb1d500686 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:38.807459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817030d7-d40a-45f0-bebf-87bb1d500686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:38.881954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-817030d7-d40a-45f0-bebf-87bb1d500686 None None] MFA Rules not processed for user `b908d5c7e7244b088fb177d2dec2731d`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:38.884986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-817030d7-d40a-45f0-bebf-87bb1d500686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:38.938156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:39.104624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 965/1930] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:39.105007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:39.267106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d83f4-f833-4270-ae5a-05cb07bf4f82 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:39.278114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67db29-7e1c-4a8f-9f89-19af30757fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:39.279506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67db29-7e1c-4a8f-9f89-19af30757fca admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:39.279506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67db29-7e1c-4a8f-9f89-19af30757fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:39.279506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67db29-7e1c-4a8f-9f89-19af30757fca admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:39.280231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67db29-7e1c-4a8f-9f89-19af30757fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:39.282575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67db29-7e1c-4a8f-9f89-19af30757fca admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:39.651449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:39.677120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 966/1931] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:39 2022] POST /identity/v3/projects => generated 381 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:39.686876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817030d7-d40a-45f0-bebf-87bb1d500686 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:39.702751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421df82-5cd8-453d-b94c-74ca4faf8c2f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:39.703168 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421df82-5cd8-453d-b94c-74ca4faf8c2f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:39.703643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421df82-5cd8-453d-b94c-74ca4faf8c2f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:39.706697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 967/1932] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:36:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:36:39.714756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817030d7-d40a-45f0-bebf-87bb1d500686 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:39.853495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:39.855377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:39.855605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:39.856031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:39.857224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:39.861452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:39.968890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:39.969534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:39.970022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:39.998618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 966/1933] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:39 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:40.007371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:40.007701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] MFA Rules not processed for user `4739aa6ea7c043d999a3280deb9963d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:40.009617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:40.023534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b849a-97cc-4817-ab73-5f9543f42c51 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:40.030239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:40.031266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:40.031758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:40.032094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:40.034298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:40.036581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:40.096529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 967/1934] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:40.103142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:40.127962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdda887-67bb-40ee-b77a-4bc36f3047f7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:40.135584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:40.136476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:40.136940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:40.137467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] PATH_INFO: `/v3/projects/e722118681424e57b7d769678adc8346/users/cbadd5cd18b64e7696d1e378ab2addf8/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:40.160961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e722118681424e57b7d769678adc8346, user_id=cbadd5cd18b64e7696d1e378ab2addf8, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:40.163816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 968/1935] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:36:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:40.166122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:40.200508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:40.202057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:40.220428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e9686-b93f-4a5a-ac23-c13dd6d10846 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:40.228256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:40.229143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:40.229598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:40.230054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:40.262206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:40.270358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:40.314661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 969/1936] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:36:40.317571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:40.324311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:40.345546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 968/1937] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:40 2022] PUT /identity/v3/projects/e722118681424e57b7d769678adc8346/users/cbadd5cd18b64e7696d1e378ab2addf8/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:40.353507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:40.372722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb074d-35f3-419e-bc1a-1383be501206 service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:40.504390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a13e-b18b-4535-8d90-2b7a342fab11 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:40.520557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:40.521166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:40.521722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:40.522057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:40.522829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:40.529100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:40.531754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:40.534565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:40.535657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] PATH_INFO: `/v3/users/e84d3ad291644b84b204b0b3102a095a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:40.536938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:40.540133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84d3ad291644b84b204b0b3102a095a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:40.545751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:40.592906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 969/1938] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:40.594553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:40.620511 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d8870-aa58-4e34-b840-4896440c003a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:40.648069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6b2db-2523-4e00-8ac2-dc029b5638a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:40.648879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6b2db-2523-4e00-8ac2-dc029b5638a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:40.649087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6b2db-2523-4e00-8ac2-dc029b5638a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:40.649360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6b2db-2523-4e00-8ac2-dc029b5638a2 admin admin] PATH_INFO: `/v3/projects/e722118681424e57b7d769678adc8346/users/cbadd5cd18b64e7696d1e378ab2addf8/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:40.653496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6b2db-2523-4e00-8ac2-dc029b5638a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e722118681424e57b7d769678adc8346, user_id=cbadd5cd18b64e7696d1e378ab2addf8, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:40.659328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6b2db-2523-4e00-8ac2-dc029b5638a2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:40.713601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 970/1939] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:40 2022] PUT /identity/v3/projects/e722118681424e57b7d769678adc8346/users/cbadd5cd18b64e7696d1e378ab2addf8/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:40.736724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:40.737795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:40.738603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:40.897313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] MFA Rules not processed for user `cbadd5cd18b64e7696d1e378ab2addf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:40.898761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:40.960147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84d3ad291644b84b204b0b3102a095a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:40.962491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84d3ad291644b84b204b0b3102a095a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:40.971641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84d3ad291644b84b204b0b3102a095a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:40.972487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84d3ad291644b84b204b0b3102a095a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:41.107129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 971/1940] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:36:41.135563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:41.137808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:41.200393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2c52985-af99-426a-bce0-b6a8288a29d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84d3ad291644b84b204b0b3102a095a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:41.246573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 970/1941] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:36:40 2022] DELETE /identity/v3/users/e84d3ad291644b84b204b0b3102a095a => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:41.253677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0853f5b-ea35-4e2d-b784-f6717fd35871 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:41.334795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:41.335439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:41.335986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:41.336272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:41.482369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:41.484099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:41.499596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 972/1942] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:36:41.525351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:41.921549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:42.134226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884ad9a-8b41-43e7-8c58-f244c3c4e62d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:42.135060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884ad9a-8b41-43e7-8c58-f244c3c4e62d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:42.135665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884ad9a-8b41-43e7-8c58-f244c3c4e62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:42.180886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7884ad9a-8b41-43e7-8c58-f244c3c4e62d None None] MFA Rules not processed for user `6e729065f1524fd9bf9ef3c6b42f7a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:42.183166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7884ad9a-8b41-43e7-8c58-f244c3c4e62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:42.249384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:42.252032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:42.497623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 971/1943] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:42.509964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d13f5-e582-4888-986f-0655194b840d service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:42.550382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:42.552416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:42.554257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:42.555437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:42.556395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] PATH_INFO: `/v3/projects/e1041161b38e4ee0b0b4d7d1f6b3994c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:42.563187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1041161b38e4ee0b0b4d7d1f6b3994c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:42.573135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:42.759486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:42.760311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:42.761131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:42.825494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1041161b38e4ee0b0b4d7d1f6b3994c', 'request_id': 'req-28525d70-ec29-4d58-8d61-905ec8f34287'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:42.841526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1041161b38e4ee0b0b4d7d1f6b3994c', 'request_id': 'req-28525d70-ec29-4d58-8d61-905ec8f34287'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:42.853704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1041161b38e4ee0b0b4d7d1f6b3994c', 'request_id': 'req-28525d70-ec29-4d58-8d61-905ec8f34287'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:42.866494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] MFA Rules not processed for user `de720e2d8a0f4b5f898e00ce90befd58`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:42.866724 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:42.987021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 972/1944] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:36:42.993427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:42.994991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:43.013010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] The token cache is being invalidate because project e1041161b38e4ee0b0b4d7d1f6b3994c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:43.021274 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:43.024378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28525d70-ec29-4d58-8d61-905ec8f34287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:43.032355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 973/1945] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:42 2022] DELETE /identity/v3/projects/e1041161b38e4ee0b0b4d7d1f6b3994c => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:43.035207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4b3f-97e0-48c1-856a-5b43f488b12f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.035716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4b3f-97e0-48c1-856a-5b43f488b12f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.036206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4b3f-97e0-48c1-856a-5b43f488b12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.079460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9a1a4b3f-97e0-48c1-856a-5b43f488b12f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:43.084087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9a1a4b3f-97e0-48c1-856a-5b43f488b12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:43.110530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5e130-ae01-4872-8b16-4b0410fb096b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.114051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 974/1946] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:43.117944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.118681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.119307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.119502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.120415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.124952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab90612-76e0-4bd4-a735-7d1b58181b2d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.125586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab90612-76e0-4bd4-a735-7d1b58181b2d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.126164 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab90612-76e0-4bd4-a735-7d1b58181b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.127282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.169723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7ab90612-76e0-4bd4-a735-7d1b58181b2d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:43.171271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7ab90612-76e0-4bd4-a735-7d1b58181b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:43.181472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 975/1947] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:43.186936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab90612-76e0-4bd4-a735-7d1b58181b2d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.199460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab90612-76e0-4bd4-a735-7d1b58181b2d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.206050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.208893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.209263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.210870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.211955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:36:43.216883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.216883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.235881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 976/1948] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:36:43.243637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.267948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96222829-756e-4c82-9c05-e76b426bb9da admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.282178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.287558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.287558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.287558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.287816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 973/1949] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:42 2022] POST /identity/v3/projects => generated 385 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:43.288917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.289160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.291174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:43.291566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.326540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:43.406615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0fca5-2259-4282-afef-5cb1b8746f42 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.414156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.415236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.415754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.416463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.417617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.422533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.423648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 977/1950] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:43 2022] POST /identity/v3/projects => generated 355 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:43.430420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.489897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:43.529731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0b43b-f5d5-49f7-9c30-0d17a7d4254d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.537802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.537802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.537802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.537802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.538542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.539440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.578046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 974/1951] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:43 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:43.580047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.597159 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd1032-a5db-4f4d-a530-2245c13a63e0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.605323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.606265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.607522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.607522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.608307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.611327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.632062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 975/1952] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:43.634309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.637070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:43.703480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 978/1953] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:43 2022] POST /identity/v3/users => generated 278 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:43.703480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.723753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf584cc-e89b-49d8-b2fd-1c943e77b1a0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.733072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.733762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.734069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.734372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] PATH_INFO: `/v3/projects/78caa9a337004170aa7cf8919813151b/users/97061137f48845398bb10504c4bf3577/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.749478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd034a9b-1be4-454a-af4f-d505b7718dd2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.758161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbadd5cd18b64e7696d1e378ab2addf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e722118681424e57b7d769678adc8346', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.759455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.760014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.760749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.761803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:36:43.762302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.765736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.783933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78caa9a337004170aa7cf8919813151b, user_id=97061137f48845398bb10504c4bf3577, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.785582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 976/1954] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:36:43.786728 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.788021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.802423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274955f0-b9e5-47d6-82fe-b6ce11ceff29 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.816744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.816744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.816744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.816744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.817182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.819526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.853821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 977/1955] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:43.854813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:43.888622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec0d3a-dc7b-4239-9a6f-619081b8d6d0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:43.894528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbadd5cd18b64e7696d1e378ab2addf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e722118681424e57b7d769678adc8346', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:43.895629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:43.896107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:43.897352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:43.898446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:43.902534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:43.942834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 979/1956] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:43 2022] PUT /identity/v3/projects/78caa9a337004170aa7cf8919813151b/users/97061137f48845398bb10504c4bf3577/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:43.943868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:44.078886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 978/1957] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:43 2022] POST /identity/v3/projects => generated 354 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:44.083597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:44.150880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611d19-42e8-42d0-aa3b-ada9b18c48b5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:44.204721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0effcc-18cb-4b89-86b8-a000a9dd4da7 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:44.212688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:44.213843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:44.214338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:44.214613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:44.215693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:44.219256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:44.241585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 979/1958] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:44.243196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:44.251736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:44.255193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:44.255193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:44.255193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:44.255193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:44.255811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:44.269781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 980/1959] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:44.275207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:44.288853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936adf4-b7b5-48a9-b1fa-f638d8eb79d4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:44.307215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:44.308973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:44.309693 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:44.310490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] PATH_INFO: `/v3/projects/78caa9a337004170aa7cf8919813151b/users/97061137f48845398bb10504c4bf3577/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:44.318039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:44.320598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78caa9a337004170aa7cf8919813151b, user_id=97061137f48845398bb10504c4bf3577, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:44.335621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:44.367740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 981/1960] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:44 2022] PUT /identity/v3/projects/78caa9a337004170aa7cf8919813151b/users/97061137f48845398bb10504c4bf3577/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:44.374696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:44.408156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c69d0-8415-4104-b876-751991a17983 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:44.487747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097ce46-4116-430c-b85b-cca580ef148e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:44.493454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518ab41-6abc-4e83-a852-746ed617b14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:44.495714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518ab41-6abc-4e83-a852-746ed617b14d admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:44.495714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518ab41-6abc-4e83-a852-746ed617b14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:44.495714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518ab41-6abc-4e83-a852-746ed617b14d admin admin] PATH_INFO: `/v3/projects/feba53df745e46a8bb9762d6560aa1ca/users/9907402c9af74b8ca2572e65895c177b/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:44.512450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518ab41-6abc-4e83-a852-746ed617b14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=feba53df745e46a8bb9762d6560aa1ca, user_id=9907402c9af74b8ca2572e65895c177b, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:44.516185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518ab41-6abc-4e83-a852-746ed617b14d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:44.531759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbadd5cd18b64e7696d1e378ab2addf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e722118681424e57b7d769678adc8346', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:44.533084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:44.533531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:44.533950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:44.535174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:36:44.535673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:44.538805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:44.557656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 980/1961] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:36:44.560551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:44.636049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:44.648492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d8cb-67b4-4f5d-b619-6bbee05f0217 tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:44.653783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 982/1962] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:44 2022] PUT /identity/v3/projects/feba53df745e46a8bb9762d6560aa1ca/users/9907402c9af74b8ca2572e65895c177b/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:44.668311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87797a-fd96-435a-beb5-0cb5ef0d9455 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:44.669250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87797a-fd96-435a-beb5-0cb5ef0d9455 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:44.669961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87797a-fd96-435a-beb5-0cb5ef0d9455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:44.717207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7a87797a-fd96-435a-beb5-0cb5ef0d9455 None None] MFA Rules not processed for user `9907402c9af74b8ca2572e65895c177b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:44.718699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7a87797a-fd96-435a-beb5-0cb5ef0d9455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:44.762455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbadd5cd18b64e7696d1e378ab2addf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e722118681424e57b7d769678adc8346', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:44.763489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:44.764267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:44.764667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] PATH_INFO: `/v3/projects/c7c0abb332c7418eacacf97af3c5e920` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:44.767235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c7c0abb332c7418eacacf97af3c5e920)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:44.776066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:44.825488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 983/1963] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:44.830806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87797a-fd96-435a-beb5-0cb5ef0d9455 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:44.963253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:44.994431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c0abb332c7418eacacf97af3c5e920', 'request_id': 'req-b7817e81-f8e9-405c-8b3a-15aec2866a2d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:45.022049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87797a-fd96-435a-beb5-0cb5ef0d9455 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:45.022294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c0abb332c7418eacacf97af3c5e920', 'request_id': 'req-b7817e81-f8e9-405c-8b3a-15aec2866a2d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:45.155845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c0abb332c7418eacacf97af3c5e920', 'request_id': 'req-b7817e81-f8e9-405c-8b3a-15aec2866a2d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:45.173561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:45.183210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] The token cache is being invalidate because project c7c0abb332c7418eacacf97af3c5e920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:45.183885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:45.187051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:45.190964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 981/1964] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:44 2022] DELETE /identity/v3/projects/c7c0abb332c7418eacacf97af3c5e920 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:45.204974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:45.205759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:45.206033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:45.206400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:45.207006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:45.210159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:45.437845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 984/1965] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:44 2022] POST /identity/v3/projects => generated 357 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:36:45.442095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:45.446095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:45.485029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7817e81-f8e9-405c-8b3a-15aec2866a2d tempest-VolumeQuotaClassesTest-1985144432 tempest-VolumeQuotaClassesTest-1985144432-project-admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:45.586592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:45.587424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:45.587706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:45.588019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:45.588857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:45.591330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:45.735541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 982/1966] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:45 2022] POST /identity/v3/users => generated 285 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:45.741656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:45.762778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e62fa-5af1-4c5d-a355-92130b7dda67 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:45.772362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:45.772893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:45.773396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:45.774506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:45.775328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:45.777218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:45.797829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 985/1967] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:45.805911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:45.830675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cef20-b5a8-4733-8f4f-2842d7697386 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:45.845341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:45.847150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:45.847150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:45.847150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] PATH_INFO: `/v3/projects/b15d0625f8644aedb5cd527698765983/users/ea52af60d54d42e4a527dff03b1abf78/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:45.871107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15d0625f8644aedb5cd527698765983, user_id=ea52af60d54d42e4a527dff03b1abf78, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:45.873818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:46.035003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 983/1968] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:45 2022] PUT /identity/v3/projects/b15d0625f8644aedb5cd527698765983/users/ea52af60d54d42e4a527dff03b1abf78/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:46.041175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:46.064160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc5f63-e64b-43fa-bdcb-a297bef2201b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:46.254212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774c63e-71cd-4a49-99a8-67446511f8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:46.254914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c63e-71cd-4a49-99a8-67446511f8ef admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:46.255203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c63e-71cd-4a49-99a8-67446511f8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:46.255533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774c63e-71cd-4a49-99a8-67446511f8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:46.257594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774c63e-71cd-4a49-99a8-67446511f8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:46.259775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774c63e-71cd-4a49-99a8-67446511f8ef admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:46.297558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 986/1969] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:46.307861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:46.332405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b975e-aee5-41d6-82ae-82811d350738 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:46.339425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:46.340137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:46.340412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:46.340925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] PATH_INFO: `/v3/projects/b15d0625f8644aedb5cd527698765983/users/ea52af60d54d42e4a527dff03b1abf78/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:46.347400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15d0625f8644aedb5cd527698765983, user_id=ea52af60d54d42e4a527dff03b1abf78, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:46.356284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:46.400483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 984/1970] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:46 2022] PUT /identity/v3/projects/b15d0625f8644aedb5cd527698765983/users/ea52af60d54d42e4a527dff03b1abf78/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:46.415446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:46.415782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:46.416018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:46.462607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:46.462607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:46.496642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] MFA Rules not processed for user `ea52af60d54d42e4a527dff03b1abf78`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:46.502066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:46.596447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d15a-ea85-4078-8da3-83d0565c66b6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:46.614014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:46.614722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:46.615025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:46.616020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] PATH_INFO: `/v3/users/b908d5c7e7244b088fb177d2dec2731d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:46.619222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 987/1971] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:46.624123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b908d5c7e7244b088fb177d2dec2731d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:46.624123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:46.643861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:46.645577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:46.695550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:46.747144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:46.775049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015403d-891a-4101-b5ee-9c096b2da1ff None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:46.783477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:46.784530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:46.785018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:46.785586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:46.809351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:46.820199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:46.835234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 988/1972] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:36:47.094667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b908d5c7e7244b088fb177d2dec2731d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:47.103503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b908d5c7e7244b088fb177d2dec2731d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:47.108678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b908d5c7e7244b088fb177d2dec2731d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:47.110160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b908d5c7e7244b088fb177d2dec2731d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:47.280578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9379d479-bf3a-43d3-9dbe-24fcf7d3dab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b908d5c7e7244b088fb177d2dec2731d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:47.372722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:47.398936 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787d6cb-66ea-4b2a-a88d-c7ccc1fef99a service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:47.405330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:47.405940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:47.406305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:47.406512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:47.410736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:47.422897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 985/1973] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:36:46 2022] DELETE /identity/v3/users/b908d5c7e7244b088fb177d2dec2731d => generated 0 bytes in 991 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:47.427801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:47.428317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:47.428764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:47.493737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:47.496933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:47.621061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:47.624807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:47.650345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 989/1974] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:36:47.661908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 986/1975] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:47.684364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:47.686221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:47.819128 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2843-457e-44aa-b2a9-43ac7179de0f service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:47.908491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:47.909398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:47.909804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:47.910182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:47.925271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:47.926821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:47.941692 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 990/1976] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:36:49.036043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:49.041883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:49.081961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196d6a1-1088-4754-939a-7fe2fcba976f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:49.091768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:49.092744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:49.093243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:49.093594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] PATH_INFO: `/v3/projects/66a50940607144d49d9036cddba73f11` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:49.095291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a50940607144d49d9036cddba73f11)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:49.098788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:49.266210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a50940607144d49d9036cddba73f11', 'request_id': 'req-f55ffcae-3024-4224-a7bf-32d860dde366'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:49.277236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a50940607144d49d9036cddba73f11', 'request_id': 'req-f55ffcae-3024-4224-a7bf-32d860dde366'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:49.389658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a50940607144d49d9036cddba73f11', 'request_id': 'req-f55ffcae-3024-4224-a7bf-32d860dde366'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:49.422328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] The token cache is being invalidate because project 66a50940607144d49d9036cddba73f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:49.422878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:49.424945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:49.429547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 987/1977] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:49 2022] DELETE /identity/v3/projects/66a50940607144d49d9036cddba73f11 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:49.433741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:49.452351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ebbb3-5183-42e2-be9d-ffc10186190b service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:49.537669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:49.573547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:49.574616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:49.575051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:49.575989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] PATH_INFO: `/v3/users/cbadd5cd18b64e7696d1e378ab2addf8` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:49.577717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbadd5cd18b64e7696d1e378ab2addf8)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:49.579956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:49.898460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbadd5cd18b64e7696d1e378ab2addf8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:49.899819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbadd5cd18b64e7696d1e378ab2addf8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:49.900608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbadd5cd18b64e7696d1e378ab2addf8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:50.017969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbadd5cd18b64e7696d1e378ab2addf8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:50.050793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12de389e-79ea-4453-ad3f-6869c6f2eeb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbadd5cd18b64e7696d1e378ab2addf8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:50.099066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 991/1978] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:36:49 2022] DELETE /identity/v3/users/cbadd5cd18b64e7696d1e378ab2addf8 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:51.494611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:51.497539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:51.535303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ffcae-3024-4224-a7bf-32d860dde366 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:51.578578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:51.708164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:51.710376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:51.711360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:51.713092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] PATH_INFO: `/v3/projects/e722118681424e57b7d769678adc8346` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:51.716266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e722118681424e57b7d769678adc8346)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:51.722086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:52.025722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e722118681424e57b7d769678adc8346', 'request_id': 'req-084f5b15-b9d3-45dc-a1a0-a059f0102487'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:52.036086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e722118681424e57b7d769678adc8346', 'request_id': 'req-084f5b15-b9d3-45dc-a1a0-a059f0102487'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:52.118867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e722118681424e57b7d769678adc8346', 'request_id': 'req-084f5b15-b9d3-45dc-a1a0-a059f0102487'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:52.150154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] The token cache is being invalidate because project e722118681424e57b7d769678adc8346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:36:52.150751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:52.151973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-084f5b15-b9d3-45dc-a1a0-a059f0102487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:36:52.154852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 988/1979] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:36:51 2022] DELETE /identity/v3/projects/e722118681424e57b7d769678adc8346 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:52.170800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f5a36-3afa-4d7c-9e83-73f2dc294194 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:52.170800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f5a36-3afa-4d7c-9e83-73f2dc294194 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:52.170800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f5a36-3afa-4d7c-9e83-73f2dc294194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:52.189111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-666f5a36-3afa-4d7c-9e83-73f2dc294194 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:52.190594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-666f5a36-3afa-4d7c-9e83-73f2dc294194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:52.325358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 992/1980] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:52.341831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a06bc4-92f2-4ddb-ba78-f56ea8e3a771 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:52.342351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a06bc4-92f2-4ddb-ba78-f56ea8e3a771 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:52.343307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a06bc4-92f2-4ddb-ba78-f56ea8e3a771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:52.376567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-19a06bc4-92f2-4ddb-ba78-f56ea8e3a771 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:52.379812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-19a06bc4-92f2-4ddb-ba78-f56ea8e3a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:52.401375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 989/1981] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:36:52.408486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f5a36-3afa-4d7c-9e83-73f2dc294194 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:52.432810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f5a36-3afa-4d7c-9e83-73f2dc294194 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:52.443023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:52.444881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:52.445263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:52.445621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:52.446590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:36:52.447044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:52.448795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:52.470941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 993/1982] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:36:52.486528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a06bc4-92f2-4ddb-ba78-f56ea8e3a771 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:52.503472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a06bc4-92f2-4ddb-ba78-f56ea8e3a771 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:52.510096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:52.511096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:52.511556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:52.511976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:52.512853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:52.524442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:52.806604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 990/1983] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:52 2022] POST /identity/v3/projects => generated 365 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:52.811091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:52.824798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbd7a1-7815-4a5a-9ce4-090534277437 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:52.929395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:52.949096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:52.949856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:52.950280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:52.950623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:52.951355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:52.954421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:53.163664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 994/1984] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:52 2022] POST /identity/v3/users => generated 283 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:53.165262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:53.180230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91997017-0bcd-45ab-bef5-4410c6f8670c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:53.202163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:53.203190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:53.203465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:53.205319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:53.206435 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:53.217347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:53.250205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 991/1985] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:53.255464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:53.290749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d747480-af97-44ae-9911-a50b6afc2807 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:53.306186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bedb91-3557-4484-a6a1-4a3820be3a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:53.307895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bedb91-3557-4484-a6a1-4a3820be3a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:53.310851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bedb91-3557-4484-a6a1-4a3820be3a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:53.311430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bedb91-3557-4484-a6a1-4a3820be3a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:53.314067 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bedb91-3557-4484-a6a1-4a3820be3a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:53.318457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bedb91-3557-4484-a6a1-4a3820be3a3c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:53.347025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 995/1986] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:36:53.349141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:53.414061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8afbd-81c2-424c-b1c2-c5c6f42290bc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:53.424201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:53.424201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:53.424201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:53.424517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] PATH_INFO: `/v3/projects/cb701a7abd5c49d6b68d0ca333f7169c/users/ecd5b9a72ef84e9d8f1636214384e37c/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:53.469210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb701a7abd5c49d6b68d0ca333f7169c, user_id=ecd5b9a72ef84e9d8f1636214384e37c, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:53.473307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:53.621483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 992/1987] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:53 2022] PUT /identity/v3/projects/cb701a7abd5c49d6b68d0ca333f7169c/users/ecd5b9a72ef84e9d8f1636214384e37c/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:36:53.623567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd641679-0cbb-48d1-8005-b87803220159 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:53.624133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd641679-0cbb-48d1-8005-b87803220159 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:53.624563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd641679-0cbb-48d1-8005-b87803220159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:53.666842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fd641679-0cbb-48d1-8005-b87803220159 None None] MFA Rules not processed for user `ecd5b9a72ef84e9d8f1636214384e37c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:53.670059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fd641679-0cbb-48d1-8005-b87803220159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:53.807814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 996/1988] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:36:53.812603 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:53.987494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eb2ce-8ec4-4826-8eb0-dc671d4507e2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:53.995025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:53.995749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:53.996033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:53.996309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:53.996958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:53.999080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:54.230463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 993/1989] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:36:53 2022] POST /identity/v3/projects => generated 367 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:54.234859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd641679-0cbb-48d1-8005-b87803220159 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:54.265244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd641679-0cbb-48d1-8005-b87803220159 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:54.404945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:54.407163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:54.407927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:54.408653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:54.419272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:54.420362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:54.620673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 997/1990] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:36:54 2022] POST /identity/v3/users => generated 290 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:36:54.626451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:54.652410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7024-9206-4aa5-b1d7-8671b2d3bcbd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:54.666679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:54.667998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:54.670920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:54.671594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:54.673796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:54.678815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:54.735489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:54.737496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 994/1991] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:36:54.739759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:54.756959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482dadf-45e9-4e0b-8f08-971a4fb5f094 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:54.766722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b4637-6abe-4193-b1ce-9cddcbcc43ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:54.767148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b4637-6abe-4193-b1ce-9cddcbcc43ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:54.767502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b4637-6abe-4193-b1ce-9cddcbcc43ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:54.767850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b4637-6abe-4193-b1ce-9cddcbcc43ad admin admin] PATH_INFO: `/v3/projects/50edc56a512c447885fe80491ba0f6d2/users/887a7468140347a9aa07c0d6313a3212/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:54.828687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b4637-6abe-4193-b1ce-9cddcbcc43ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=50edc56a512c447885fe80491ba0f6d2, user_id=887a7468140347a9aa07c0d6313a3212, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:54.834816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b4637-6abe-4193-b1ce-9cddcbcc43ad admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:54.997552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 998/1992] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:54 2022] PUT /identity/v3/projects/50edc56a512c447885fe80491ba0f6d2/users/887a7468140347a9aa07c0d6313a3212/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:55.011434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:55.023866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:55.024191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:55.024992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:55.032770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f085f0-4938-43d3-8e3d-d867555c4776 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:55.078297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] MFA Rules not processed for user `97061137f48845398bb10504c4bf3577`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:55.080338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:55.166897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:55.167676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:55.168036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:55.168393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:55.169221 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:55.173088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:55.189633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 995/1993] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:36:55.199928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:55.232023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84499397-0b61-42ac-b941-9726b164afb4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:55.238198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 999/1994] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:36:55.280089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:55.280089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:55.280707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18b669-c8b8-4498-bc14-58e85710a3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:55.280707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18b669-c8b8-4498-bc14-58e85710a3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:55.280707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18b669-c8b8-4498-bc14-58e85710a3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:55.280707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18b669-c8b8-4498-bc14-58e85710a3e7 admin admin] PATH_INFO: `/v3/projects/50edc56a512c447885fe80491ba0f6d2/users/887a7468140347a9aa07c0d6313a3212/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:55.284567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18b669-c8b8-4498-bc14-58e85710a3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50edc56a512c447885fe80491ba0f6d2, user_id=887a7468140347a9aa07c0d6313a3212, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:55.299217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18b669-c8b8-4498-bc14-58e85710a3e7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:55.328001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 996/1995] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:36:55 2022] PUT /identity/v3/projects/50edc56a512c447885fe80491ba0f6d2/users/887a7468140347a9aa07c0d6313a3212/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:36:55.339411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:55.339694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:55.339961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:55.447033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] MFA Rules not processed for user `887a7468140347a9aa07c0d6313a3212`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:55.450498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:55.520289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02514e6d-a130-44b2-add1-4c4ac9657c80 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:55.638557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3d715-f710-4793-b308-90c176e23858 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:55.641169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3d715-f710-4793-b308-90c176e23858 service glance] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:55.641469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3d715-f710-4793-b308-90c176e23858 service glance] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:55.645583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3d715-f710-4793-b308-90c176e23858 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:55.648716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-eef3d715-f710-4793-b308-90c176e23858 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:55.676342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 997/1996] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:36:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:36:55.744292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3d715-f710-4793-b308-90c176e23858 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:55.746418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3d715-f710-4793-b308-90c176e23858 service glance] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:55.759747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1000/1997] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:36:55.996048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:36:56.376126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:56.406704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:56.543573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8a74-d462-4bbd-bfe0-1b70c24768be None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:56.548057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:56.549016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:56.549483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:56.550150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:56.550524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5743953-c56b-4615-8ac5-da73e9a359f8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:56.550990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5743953-c56b-4615-8ac5-da73e9a359f8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:56.551093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5743953-c56b-4615-8ac5-da73e9a359f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:56.563205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:56.565106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:56.579433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 998/1998] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:36:56.603734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b5743953-c56b-4615-8ac5-da73e9a359f8 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:36:56.605316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b5743953-c56b-4615-8ac5-da73e9a359f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:56.791945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1001/1999] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:36:56.893280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:56.896878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:36:57.115960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39770ad4-35cb-449d-8197-d3e5e3af6239 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:57.121930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c89d6-2272-462d-8eab-c5e84b4a8e65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:57.122832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c89d6-2272-462d-8eab-c5e84b4a8e65 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:57.123303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c89d6-2272-462d-8eab-c5e84b4a8e65 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:57.123649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c89d6-2272-462d-8eab-c5e84b4a8e65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:57.136555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c89d6-2272-462d-8eab-c5e84b4a8e65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:57.138234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c89d6-2272-462d-8eab-c5e84b4a8e65 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:57.151507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 999/2000] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:36:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:36:57.312310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5743953-c56b-4615-8ac5-da73e9a359f8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:36:57.335346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5743953-c56b-4615-8ac5-da73e9a359f8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:36:57.356106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:36:57.357588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:36:57.358106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:36:57.358603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:36:57.553583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:36:57.556440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:36:57.598838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1002/2001] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:36:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:36:59.529536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:00.439512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:12.245405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77c6a4-2937-48e8-a498-a1836177902f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:12.248134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77c6a4-2937-48e8-a498-a1836177902f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:12.248947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77c6a4-2937-48e8-a498-a1836177902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:12.304988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fc77c6a4-2937-48e8-a498-a1836177902f None None] MFA Rules not processed for user `34b1955a937f4053967cf82cedd87a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:12.305891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=55 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:12.311694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fc77c6a4-2937-48e8-a498-a1836177902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:12.501414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1000/2002] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:13.861420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:14.040719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262bce6-9308-4f3e-8992-8a4b0bd94bca service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:14.049878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:14.051014 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:14.051656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:14.052133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] PATH_INFO: `/v3/users/ecd5b9a72ef84e9d8f1636214384e37c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:14.055964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd5b9a72ef84e9d8f1636214384e37c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:14.060358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:14.466730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd5b9a72ef84e9d8f1636214384e37c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:14.469793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd5b9a72ef84e9d8f1636214384e37c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:14.471827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd5b9a72ef84e9d8f1636214384e37c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:14.589409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd5b9a72ef84e9d8f1636214384e37c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:14.621470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd5b9a72ef84e9d8f1636214384e37c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:14.675266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1003/2003] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:13 2022] DELETE /identity/v3/users/ecd5b9a72ef84e9d8f1636214384e37c => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:14.695172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:14.695172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:14.695172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:14.741863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:14.750846 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:14.924629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1001/2004] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:14.945495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:14.947220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:15.040034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41e0c0-050d-40b5-a916-63900922b874 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:15.045966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d0589-a655-4691-991f-eeb14d6e28d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:15.046833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d0589-a655-4691-991f-eeb14d6e28d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:15.047298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d0589-a655-4691-991f-eeb14d6e28d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:15.047752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d0589-a655-4691-991f-eeb14d6e28d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:15.062230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d0589-a655-4691-991f-eeb14d6e28d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:15.063779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d0589-a655-4691-991f-eeb14d6e28d3 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:15.080303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1004/2005] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:15.937401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:15.940384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:15.968023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c6812-5b84-49ac-9fea-498a9f1838a7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:15.989069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:15.990285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:15.990823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:15.993538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] PATH_INFO: `/v3/projects/cb701a7abd5c49d6b68d0ca333f7169c` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:15.996684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb701a7abd5c49d6b68d0ca333f7169c)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:16.000131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:16.003358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:16.181676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441983f-0cc5-44c8-83ee-423a2aa5a3b7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:16.183278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441983f-0cc5-44c8-83ee-423a2aa5a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:16.184013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441983f-0cc5-44c8-83ee-423a2aa5a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:16.217571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6441983f-0cc5-44c8-83ee-423a2aa5a3b7 None None] MFA Rules not processed for user `de720e2d8a0f4b5f898e00ce90befd58`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:16.219642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6441983f-0cc5-44c8-83ee-423a2aa5a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:16.221630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb701a7abd5c49d6b68d0ca333f7169c', 'request_id': 'req-660861c9-f726-4fc4-bf51-4224ceeea788'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:16.242226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb701a7abd5c49d6b68d0ca333f7169c', 'request_id': 'req-660861c9-f726-4fc4-bf51-4224ceeea788'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:16.247941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:16.304857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1005/2006] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:16.419210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb701a7abd5c49d6b68d0ca333f7169c', 'request_id': 'req-660861c9-f726-4fc4-bf51-4224ceeea788'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:16.449854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] The token cache is being invalidate because project cb701a7abd5c49d6b68d0ca333f7169c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:16.449974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:16.453478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-660861c9-f726-4fc4-bf51-4224ceeea788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:16.462285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1002/2007] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:15 2022] DELETE /identity/v3/projects/cb701a7abd5c49d6b68d0ca333f7169c => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:37:16.465735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441983f-0cc5-44c8-83ee-423a2aa5a3b7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:16.647006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:16.651606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:16.652816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:16.685045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441983f-0cc5-44c8-83ee-423a2aa5a3b7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:16.697048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] MFA Rules not processed for user `3570472f8c2b42b480ec1d1220414bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:16.698669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:16.711874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:16.713150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:16.713633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:16.714120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] PATH_INFO: `/v3/users/887a7468140347a9aa07c0d6313a3212` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:16.718489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] RBAC: Authorizing `identity:delete_user(user_id=887a7468140347a9aa07c0d6313a3212)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:16.724488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:16.828895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1003/2008] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:17.098360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '887a7468140347a9aa07c0d6313a3212'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:17.100369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '887a7468140347a9aa07c0d6313a3212'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:17.101243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '887a7468140347a9aa07c0d6313a3212'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:17.287604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '887a7468140347a9aa07c0d6313a3212'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:17.322330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-12f24b30-0b21-4b4c-b223-d46928a9708c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '887a7468140347a9aa07c0d6313a3212'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:17.351338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1006/2009] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:16 2022] DELETE /identity/v3/users/887a7468140347a9aa07c0d6313a3212 => generated 0 bytes in 886 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:37:18.352648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:18.354633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:18.394860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52946b04-f936-4e69-a779-3b6d9566b640 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:18.487493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:18.488579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:18.488579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:18.489336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] PATH_INFO: `/v3/projects/50edc56a512c447885fe80491ba0f6d2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:18.491390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50edc56a512c447885fe80491ba0f6d2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:18.493435 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:18.692370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:18.703000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50edc56a512c447885fe80491ba0f6d2', 'request_id': 'req-9544a1f7-3c5e-474e-af10-28efe6e1b876'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:18.717395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50edc56a512c447885fe80491ba0f6d2', 'request_id': 'req-9544a1f7-3c5e-474e-af10-28efe6e1b876'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:18.834531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50edc56a512c447885fe80491ba0f6d2', 'request_id': 'req-9544a1f7-3c5e-474e-af10-28efe6e1b876'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:18.857174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] The token cache is being invalidate because project 50edc56a512c447885fe80491ba0f6d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:18.857883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:18.859427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544a1f7-3c5e-474e-af10-28efe6e1b876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:18.862581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1004/2010] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:18 2022] DELETE /identity/v3/projects/50edc56a512c447885fe80491ba0f6d2 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:18.870784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a205bde-6e03-4c24-bcae-6d8d6e68fd25 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:18.871591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a205bde-6e03-4c24-bcae-6d8d6e68fd25 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:18.871784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a205bde-6e03-4c24-bcae-6d8d6e68fd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:18.913885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2a205bde-6e03-4c24-bcae-6d8d6e68fd25 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:18.917065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2a205bde-6e03-4c24-bcae-6d8d6e68fd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:19.039982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:19.097611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1007/2011] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:19.106261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62821-0e2c-4314-b838-ff2bdece5ec9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.106616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62821-0e2c-4314-b838-ff2bdece5ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.106984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62821-0e2c-4314-b838-ff2bdece5ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.144635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3bd62821-0e2c-4314-b838-ff2bdece5ec9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:19.147493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3bd62821-0e2c-4314-b838-ff2bdece5ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:19.177973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1005/2012] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:19.179365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a205bde-6e03-4c24-bcae-6d8d6e68fd25 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:19.194233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a205bde-6e03-4c24-bcae-6d8d6e68fd25 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:19.200927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:19.201563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.202035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.202308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.202972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:37:19.203279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:19.205109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:19.224635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1008/2013] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:37:19.229274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd62821-0e2c-4314-b838-ff2bdece5ec9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:19.246254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd62821-0e2c-4314-b838-ff2bdece5ec9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:19.252271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:19.253194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.253766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.254233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.255274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:19.258181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:19.422089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1006/2014] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:19 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:37:19.426569 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:19.438422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8574286-4088-4fcb-8c92-5655e209373e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:19.511591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:19.512231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.512513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.512786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.513475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:19.515526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:19.623675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1009/2015] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:19 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:37:19.628259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:19.647064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b98a8-ab80-4940-99b3-4de4578bca35 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:19.655455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:19.656549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.657049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.657534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.660302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:19.663037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:19.683992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1007/2016] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:19.688166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:19.710075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8add2-6baf-4a93-bdc7-1c45807da169 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:19.715517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a76bb6-6908-4ccc-810c-6580d38ed6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:19.716302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76bb6-6908-4ccc-810c-6580d38ed6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.716636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76bb6-6908-4ccc-810c-6580d38ed6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.716957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76bb6-6908-4ccc-810c-6580d38ed6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.717623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a76bb6-6908-4ccc-810c-6580d38ed6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:19.719436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a76bb6-6908-4ccc-810c-6580d38ed6bf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:19.743689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1010/2017] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:19.745582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:19.762814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb1da-59c2-4141-b847-4157b45b1169 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:19.769200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:19.769921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.770230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.770535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] PATH_INFO: `/v3/projects/eeaf18f1e833457797fb4f0ceb2900b5/users/55a8e283f3f140ffb9d5ffcf0772eaa9/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.787963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeaf18f1e833457797fb4f0ceb2900b5, user_id=55a8e283f3f140ffb9d5ffcf0772eaa9, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:19.790755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:19.918343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1008/2018] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:19 2022] PUT /identity/v3/projects/eeaf18f1e833457797fb4f0ceb2900b5/users/55a8e283f3f140ffb9d5ffcf0772eaa9/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:19.938117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a295e-b5d3-4254-b1c3-ac0f07f5e74f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:19.938678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a295e-b5d3-4254-b1c3-ac0f07f5e74f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:19.939054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a295e-b5d3-4254-b1c3-ac0f07f5e74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:19.976986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-963a295e-b5d3-4254-b1c3-ac0f07f5e74f None None] MFA Rules not processed for user `55a8e283f3f140ffb9d5ffcf0772eaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:19.978625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-963a295e-b5d3-4254-b1c3-ac0f07f5e74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:20.076814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1011/2019] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:20.089522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:20.270532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b0716-af4d-4846-8fb0-4a388a45b816 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:20.283011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:20.284033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:20.285225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:20.285225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:20.286145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:20.291919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:20.508161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1009/2020] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:20 2022] POST /identity/v3/projects => generated 381 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:20.513663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a295e-b5d3-4254-b1c3-ac0f07f5e74f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:20.537334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a295e-b5d3-4254-b1c3-ac0f07f5e74f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:20.663724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:20.664798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:20.665269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:20.665741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:20.667044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:20.671452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:20.875287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1012/2021] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:20 2022] POST /identity/v3/users => generated 297 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:20.877296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:20.899180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae0f61-3f5f-405d-9fd2-2e67ce378fcc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:20.908705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903e09f-69b5-4fd1-ac38-956faa33ef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:20.910362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903e09f-69b5-4fd1-ac38-956faa33ef01 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:20.911035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903e09f-69b5-4fd1-ac38-956faa33ef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:20.911756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903e09f-69b5-4fd1-ac38-956faa33ef01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:20.913315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903e09f-69b5-4fd1-ac38-956faa33ef01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:20.916734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903e09f-69b5-4fd1-ac38-956faa33ef01 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:20.954922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1010/2022] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:37:20.960682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:20.973818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abcbd0-d4c7-40a7-bb22-afd4ce7b986e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:20.979847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:20.980789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:20.981114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:20.981531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] PATH_INFO: `/v3/projects/6bdb9a2b78f945508183957f616e20a6/users/8ed6039cec424448baea40264d85ecbd/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:20.993001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:20.993456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:20.993908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:21.008209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bdb9a2b78f945508183957f616e20a6, user_id=8ed6039cec424448baea40264d85ecbd, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:21.013973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:21.026979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:21.028057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:21.145597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1013/2023] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:20 2022] PUT /identity/v3/projects/6bdb9a2b78f945508183957f616e20a6/users/8ed6039cec424448baea40264d85ecbd/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:21.151822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:21.158211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1011/2024] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:37:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:21.170768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:21.172296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:21.228474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:21.282612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b4a2-324a-4e03-ae79-feb12e968f0f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:21.292546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:21.293605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:21.294182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:21.294831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:21.296130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:21.299250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:21.312443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd1230-1f6c-4aac-adac-b161707b3f24 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:21.336302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1014/2025] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:21.341727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:21.355473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde0fbd-f2df-484f-8bab-0a68c87b4cf5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:21.362828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef2bba-8471-4fb0-a983-f057d84a3aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:21.364292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef2bba-8471-4fb0-a983-f057d84a3aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:21.368313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef2bba-8471-4fb0-a983-f057d84a3aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:21.371019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef2bba-8471-4fb0-a983-f057d84a3aed admin admin] PATH_INFO: `/v3/projects/6bdb9a2b78f945508183957f616e20a6/users/8ed6039cec424448baea40264d85ecbd/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:21.375746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef2bba-8471-4fb0-a983-f057d84a3aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bdb9a2b78f945508183957f616e20a6, user_id=8ed6039cec424448baea40264d85ecbd, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:21.398223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef2bba-8471-4fb0-a983-f057d84a3aed admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:21.422344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1015/2026] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:21 2022] PUT /identity/v3/projects/6bdb9a2b78f945508183957f616e20a6/users/8ed6039cec424448baea40264d85ecbd/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:21.435369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:21.438455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:21.438455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:21.465601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:21.466993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:21.467650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:21.468302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:21.520476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] MFA Rules not processed for user `8ed6039cec424448baea40264d85ecbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:21.524504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:21.630134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:21.633023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:21.636435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1016/2027] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:21.653587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:21.656628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:21.658829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1012/2028] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:37:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:37:21.659757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:21.820664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b48b2-03ee-41f4-bae2-2e4a84993e6e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:21.839745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:21.839745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:21.839745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:21.839745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:21.886048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:21.889789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:21.919036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1017/2029] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:23.155347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:23.557504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:23.557504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:23.616478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b300d-282a-40c8-abd7-5435890cefd9 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:23.651042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:23.727941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:23.777005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5ca74-d078-4a9b-bfc1-be04bfb89e03 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:23.790752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:23.791437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:23.792013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:23.792254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] PATH_INFO: `/v3/users/55a8e283f3f140ffb9d5ffcf0772eaa9` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:23.795026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a8e283f3f140ffb9d5ffcf0772eaa9)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:23.797544 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:23.847158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:23.848425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:23.849018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:23.849661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] PATH_INFO: `/v3/users/9907402c9af74b8ca2572e65895c177b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:23.851838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9907402c9af74b8ca2572e65895c177b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:23.855212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:24.169253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a8e283f3f140ffb9d5ffcf0772eaa9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.170647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a8e283f3f140ffb9d5ffcf0772eaa9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.173653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a8e283f3f140ffb9d5ffcf0772eaa9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.215205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9907402c9af74b8ca2572e65895c177b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.216286 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9907402c9af74b8ca2572e65895c177b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.237366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9907402c9af74b8ca2572e65895c177b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.241276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9907402c9af74b8ca2572e65895c177b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.305160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a8e283f3f140ffb9d5ffcf0772eaa9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.380314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-455e1a82-63b7-4d84-b19e-05c2c3bd7685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a8e283f3f140ffb9d5ffcf0772eaa9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.400006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b11e84d1-e932-4428-a62c-6fb536ff51e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9907402c9af74b8ca2572e65895c177b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:24.438198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1013/2030] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:23 2022] DELETE /identity/v3/users/9907402c9af74b8ca2572e65895c177b => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:24.447000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6454-c548-4d1c-84f7-74d241639924 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:24.447000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6454-c548-4d1c-84f7-74d241639924 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:24.447000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6454-c548-4d1c-84f7-74d241639924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:24.495648 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a47d6454-c548-4d1c-84f7-74d241639924 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:24.497244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a47d6454-c548-4d1c-84f7-74d241639924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:24.505768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1018/2031] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:23 2022] DELETE /identity/v3/users/55a8e283f3f140ffb9d5ffcf0772eaa9 => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:24.511660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f2681-6171-4c94-b924-ec520249803b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:24.514519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f2681-6171-4c94-b924-ec520249803b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:24.515498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f2681-6171-4c94-b924-ec520249803b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:24.601651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c30f2681-6171-4c94-b924-ec520249803b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:24.609807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c30f2681-6171-4c94-b924-ec520249803b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:24.667998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1014/2032] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:24.670025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:24.670501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:24.670849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:24.699293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:24.752902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:24.760880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1019/2033] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:24.761837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:24.765691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f2681-6171-4c94-b924-ec520249803b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:24.767463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c30f2681-6171-4c94-b924-ec520249803b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:24.810142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1015/2034] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:24.811805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:24.820892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:24.940542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f2681-6171-4c94-b924-ec520249803b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:24.971073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:24.971073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:24.971548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:24.971892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:24.984247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:25.002110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:25.003968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:25.013253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f646a-286f-42bc-ba82-b2ebf94567b2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:25.023208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:25.023208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:25.023208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:25.023208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:25.033518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1020/2035] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:37:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:37:25.034626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:25.051066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:25.054983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:25.077863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee68fb-ef52-456f-9301-363ed82b4a44 service swift] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:25.081750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1016/2036] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:25.087571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:25.087571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:25.092831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:25.093513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:25.125191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:25.131020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:25.161048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1021/2037] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:26.994069 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:26.996350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:27.065149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5842a-2213-4b34-9273-a3a367f16093 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:27.097245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:27.098295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:27.098801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:27.099337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] PATH_INFO: `/v3/projects/feba53df745e46a8bb9762d6560aa1ca` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:27.104940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feba53df745e46a8bb9762d6560aa1ca)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:27.117583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:27.440672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feba53df745e46a8bb9762d6560aa1ca', 'request_id': 'req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:27.488075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feba53df745e46a8bb9762d6560aa1ca', 'request_id': 'req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:27.577974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feba53df745e46a8bb9762d6560aa1ca', 'request_id': 'req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:27.628222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] The token cache is being invalidate because project feba53df745e46a8bb9762d6560aa1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:27.628744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:27.631256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:27.638475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1017/2038] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:26 2022] DELETE /identity/v3/projects/feba53df745e46a8bb9762d6560aa1ca => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:27.651906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:27.667006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa7fd8-a542-464a-ae22-134d874bd4c2 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:27.885464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:27.889937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:27.892207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:27.893079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] PATH_INFO: `/v3/users/ea52af60d54d42e4a527dff03b1abf78` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:27.901941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea52af60d54d42e4a527dff03b1abf78)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:27.909979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:28.248905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:28.251090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:28.340170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1112c-9651-4ff8-bcda-ab1d97ee92d3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:28.342134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea52af60d54d42e4a527dff03b1abf78'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:28.345433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea52af60d54d42e4a527dff03b1abf78'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:28.347830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea52af60d54d42e4a527dff03b1abf78'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:28.403420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:28.413311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:28.413311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:28.413311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] PATH_INFO: `/v3/projects/eeaf18f1e833457797fb4f0ceb2900b5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:28.415340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeaf18f1e833457797fb4f0ceb2900b5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:28.419583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:28.526463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea52af60d54d42e4a527dff03b1abf78'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:28.531367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:28.587587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea52af60d54d42e4a527dff03b1abf78'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:28.673974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1022/2039] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:27 2022] DELETE /identity/v3/users/ea52af60d54d42e4a527dff03b1abf78 => generated 0 bytes in 1022 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:28.840795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeaf18f1e833457797fb4f0ceb2900b5', 'request_id': 'req-f317f1fe-47b9-4783-81f3-0a7890bc35e6'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:28.865271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeaf18f1e833457797fb4f0ceb2900b5', 'request_id': 'req-f317f1fe-47b9-4783-81f3-0a7890bc35e6'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:29.006845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeaf18f1e833457797fb4f0ceb2900b5', 'request_id': 'req-f317f1fe-47b9-4783-81f3-0a7890bc35e6'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:29.069499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] The token cache is being invalidate because project eeaf18f1e833457797fb4f0ceb2900b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:29.070124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:29.076822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:29.095235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1018/2040] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:28 2022] DELETE /identity/v3/projects/eeaf18f1e833457797fb4f0ceb2900b5 => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:29.096038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:29.097546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:29.266778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd87d-1969-40c5-8a4f-313b58ff4735 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:29.545927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:29.545927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:29.545927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:29.545927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] PATH_INFO: `/v3/users/8ed6039cec424448baea40264d85ecbd` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:29.546596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed6039cec424448baea40264d85ecbd)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:29.546870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:30.031579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed6039cec424448baea40264d85ecbd'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:30.032663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed6039cec424448baea40264d85ecbd'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:30.033247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed6039cec424448baea40264d85ecbd'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:30.165826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed6039cec424448baea40264d85ecbd'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:30.175694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c1690d91-ad5d-484f-91f1-1cc2120b3a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed6039cec424448baea40264d85ecbd'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:30.249604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:30.264620 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1023/2041] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:29 2022] DELETE /identity/v3/users/8ed6039cec424448baea40264d85ecbd => generated 0 bytes in 1166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:30.617219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:30.618854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:30.656431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317f1fe-47b9-4783-81f3-0a7890bc35e6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:30.892312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:30.924189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:30.929183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:30.930033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:30.930675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] PATH_INFO: `/v3/projects/b15d0625f8644aedb5cd527698765983` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:30.933139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b15d0625f8644aedb5cd527698765983)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:30.937867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:31.189956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b15d0625f8644aedb5cd527698765983', 'request_id': 'req-60565941-d719-4c3d-a1e3-00b8a3bd37dc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:31.205339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b15d0625f8644aedb5cd527698765983', 'request_id': 'req-60565941-d719-4c3d-a1e3-00b8a3bd37dc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:31.303723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b15d0625f8644aedb5cd527698765983', 'request_id': 'req-60565941-d719-4c3d-a1e3-00b8a3bd37dc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:31.378595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] The token cache is being invalidate because project b15d0625f8644aedb5cd527698765983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:31.383133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:31.401618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-60565941-d719-4c3d-a1e3-00b8a3bd37dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:31.406635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1019/2042] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:30 2022] DELETE /identity/v3/projects/b15d0625f8644aedb5cd527698765983 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:37:31.416627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581806d9-05ed-4063-9b9d-e50e4bc12140 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:31.416627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581806d9-05ed-4063-9b9d-e50e4bc12140 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:31.416627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581806d9-05ed-4063-9b9d-e50e4bc12140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:31.461286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-581806d9-05ed-4063-9b9d-e50e4bc12140 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:31.463173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-581806d9-05ed-4063-9b9d-e50e4bc12140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:31.659177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1024/2043] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:31.673060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee663-13f3-4773-81e5-24df73db6953 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:31.673922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee663-13f3-4773-81e5-24df73db6953 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:31.674569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee663-13f3-4773-81e5-24df73db6953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:31.748456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f0dee663-13f3-4773-81e5-24df73db6953 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:31.750803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f0dee663-13f3-4773-81e5-24df73db6953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:31.795368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581806d9-05ed-4063-9b9d-e50e4bc12140 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:31.796113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1020/2044] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:31.864146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581806d9-05ed-4063-9b9d-e50e4bc12140 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:31.887556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:31.888738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:31.890357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:31.890671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:31.891379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:37:31.891831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:31.896814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:31.933372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1025/2045] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:37:31.947694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee663-13f3-4773-81e5-24df73db6953 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:31.949768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee663-13f3-4773-81e5-24df73db6953 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:31.964871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:31.965922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:31.966408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:31.970357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:31.971458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:31.982579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:32.930995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:32.938156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1021/2046] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:31 2022] POST /identity/v3/projects => generated 367 bytes in 998 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:32.939073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:32.945764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:32.977806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d3512-6821-4a36-bd4d-c2f3ad0c36a0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:33.163943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082949-f26e-4843-bd97-620f263e33ad admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:33.166471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:33.167429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.169229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.169430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:33.169685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] PATH_INFO: `/v3/projects/6bdb9a2b78f945508183957f616e20a6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.170032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.170313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.172765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.172765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:33.173375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:33.187586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bdb9a2b78f945508183957f616e20a6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:33.190036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:33.289516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1022/2047] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:32 2022] POST /identity/v3/users => generated 284 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:33.298575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:33.348840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffb989-2779-40a8-8c7f-a3ed7076010e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:33.363868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:33.367448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.367756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.368077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.369239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:33.380843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:33.422120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1023/2048] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:33.424188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:33.459740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff18df-c38d-4e91-80fd-8850356190af admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:33.482917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bdb9a2b78f945508183957f616e20a6', 'request_id': 'req-34c10965-3f8a-4fc5-b00f-8766e896b9ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:33.515057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bdb9a2b78f945508183957f616e20a6', 'request_id': 'req-34c10965-3f8a-4fc5-b00f-8766e896b9ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:33.535448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bdb9a2b78f945508183957f616e20a6', 'request_id': 'req-34c10965-3f8a-4fc5-b00f-8766e896b9ed'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:33.575089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:33.657909 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] The token cache is being invalidate because project 6bdb9a2b78f945508183957f616e20a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:33.661765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:33.668166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-34c10965-3f8a-4fc5-b00f-8766e896b9ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:33.677919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1026/2049] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:32 2022] DELETE /identity/v3/projects/6bdb9a2b78f945508183957f616e20a6 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:33.688578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2547e7-ecad-4f64-949e-170baee941cf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.688958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2547e7-ecad-4f64-949e-170baee941cf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.695786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2547e7-ecad-4f64-949e-170baee941cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.716429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:33.717504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.718200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.718723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.719646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:33.731492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:33.750063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0e2547e7-ecad-4f64-949e-170baee941cf None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:33.760241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0e2547e7-ecad-4f64-949e-170baee941cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:33.770389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1024/2050] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:33.773001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:33.788845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1027/2051] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:33.794732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:33.809048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e13a2-9462-47b2-9ff2-34c3b7440a57 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.809377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e13a2-9462-47b2-9ff2-34c3b7440a57 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.809642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e13a2-9462-47b2-9ff2-34c3b7440a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.836627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee3579-fcb7-4120-a038-87a3df27f77b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:33.857973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e87f8-8e46-46d2-b903-c2266d108d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:33.858479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e87f8-8e46-46d2-b903-c2266d108d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.858765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e87f8-8e46-46d2-b903-c2266d108d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.859054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e87f8-8e46-46d2-b903-c2266d108d13 admin admin] PATH_INFO: `/v3/projects/c5d89faecd2449c4b3b51fe739df15f1/users/52fa5939b312450daf12612d891cbc33/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.869466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-705e13a2-9462-47b2-9ff2-34c3b7440a57 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:33.870755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-705e13a2-9462-47b2-9ff2-34c3b7440a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:33.911086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1028/2052] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:33.912247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e13a2-9462-47b2-9ff2-34c3b7440a57 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:33.916880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e87f8-8e46-46d2-b903-c2266d108d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d89faecd2449c4b3b51fe739df15f1, user_id=52fa5939b312450daf12612d891cbc33, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:33.919796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e87f8-8e46-46d2-b903-c2266d108d13 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:33.938550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:33.952102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e13a2-9462-47b2-9ff2-34c3b7440a57 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:33.978751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:33.979765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:33.980233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:33.980702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:33.981873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:37:33.982412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:33.985470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:34.047215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1029/2053] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:37:34.051360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1025/2054] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:33 2022] PUT /identity/v3/projects/c5d89faecd2449c4b3b51fe739df15f1/users/52fa5939b312450daf12612d891cbc33/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:34.078101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:34.082184 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7083-6f8d-49b8-92e6-54392447b980 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:34.082455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7083-6f8d-49b8-92e6-54392447b980 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:34.082828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7083-6f8d-49b8-92e6-54392447b980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:34.124276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750daf6c-d599-4fbb-944a-348847bc13d8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:34.165869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-799b7083-6f8d-49b8-92e6-54392447b980 None None] MFA Rules not processed for user `52fa5939b312450daf12612d891cbc33`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:34.177749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-799b7083-6f8d-49b8-92e6-54392447b980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:34.378202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:34.379555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:34.380467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:34.381417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:34.385908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:34.403134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:34.483162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1026/2055] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:34.484954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7083-6f8d-49b8-92e6-54392447b980 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:34.540424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7083-6f8d-49b8-92e6-54392447b980 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:34.560933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:34.564251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:34.564881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:34.566318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:34.568533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:34.597473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:34.663635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1030/2056] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:34 2022] POST /identity/v3/projects => generated 371 bytes in 588 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:34.665866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:34.845105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1027/2057] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:34 2022] POST /identity/v3/projects => generated 369 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:34.850738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:34.914052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf1f6-c11b-4a0e-b6e5-1f7eef545b1f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:35.150034 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4350fa-1bcd-4bfa-bd99-cfeca2f87f15 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:35.177608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:35.178494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:35.178776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:35.179052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:35.179714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:35.181737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:35.207059 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:35.207864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:35.208006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:35.208239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:35.209412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:35.211466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:35.368806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1028/2058] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:34 2022] POST /identity/v3/users => generated 291 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:35.374379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:35.374680 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1031/2059] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:34 2022] POST /identity/v3/users => generated 286 bytes in 710 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:35.375415 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:35.404689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:35.413401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345945c2-a6aa-4387-a408-ad3a5d3a86aa admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:35.414368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad23e7-dd29-4819-b622-a7d9cbbc5bb7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:35.434024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:35.445138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:35.445138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:35.445138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:35.445138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:35.445138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:35.450816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:35.451789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:35.452296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:35.452748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:35.453890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:35.456787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:35.480944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1029/2060] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:35.486950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:35.517709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528265e4-f80d-44f3-98cb-a508f3719a98 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:35.532301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1032/2061] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:35.533765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:35.535398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:35.536820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:35.537236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] PATH_INFO: `/v3/projects/2a4953389e354347972e63683cb97a8a/users/b91c3e3a467b4f40a35551e2494c8180/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:35.542750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:35.573334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:35.608282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806d951-2ac6-443b-a0b8-02cc2fb5a295 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:35.621757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4953389e354347972e63683cb97a8a, user_id=b91c3e3a467b4f40a35551e2494c8180, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:35.626372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:35.627694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:35.628003 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:35.628277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:35.628984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:35.634788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:35.636296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:35.682831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1033/2062] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:35.685370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:35.722974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fcf1e-baa2-4753-b959-16165f78b676 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:35.751702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43c5f4-c7f2-4912-8ab2-e0215c13a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:35.752579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c5f4-c7f2-4912-8ab2-e0215c13a567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:35.752866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c5f4-c7f2-4912-8ab2-e0215c13a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:35.761687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c5f4-c7f2-4912-8ab2-e0215c13a567 admin admin] PATH_INFO: `/v3/projects/d0d567af87e941398ce937433dae18c7/users/043258ad0f2b4ec0a5217cee4429d68b/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:35.795471 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1030/2063] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:35 2022] PUT /identity/v3/projects/2a4953389e354347972e63683cb97a8a/users/b91c3e3a467b4f40a35551e2494c8180/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:35.810973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:35.828436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43c5f4-c7f2-4912-8ab2-e0215c13a567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d567af87e941398ce937433dae18c7, user_id=043258ad0f2b4ec0a5217cee4429d68b, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:35.841117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43c5f4-c7f2-4912-8ab2-e0215c13a567 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:35.989760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadd7d-6eb4-45a6-9ec3-e86d00a47e3e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:36.014109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:36.015769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:36.016401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:36.016918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:36.024535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1034/2064] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:35 2022] PUT /identity/v3/projects/d0d567af87e941398ce937433dae18c7/users/043258ad0f2b4ec0a5217cee4429d68b/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:36.025051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:36.025051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:36.026256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:36.026547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:36.026827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:36.061257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1031/2065] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:36.064671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:36.070422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:36.071804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:36.089070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4be58f-6f51-4939-9357-8e2d3d5b402d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:36.265510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1035/2066] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:36.267539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:36.269914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:36.499747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:36.529588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:36.533932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:36.535209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] PATH_INFO: `/v3/projects/2a4953389e354347972e63683cb97a8a/users/b91c3e3a467b4f40a35551e2494c8180/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:36.556250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4953389e354347972e63683cb97a8a, user_id=b91c3e3a467b4f40a35551e2494c8180, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:36.576663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f560d-e259-4b54-b9f6-b999e68eb6f0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:36.580737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:36.584949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eab46c-670a-42a4-8b31-cdcc4857b9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:36.585640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eab46c-670a-42a4-8b31-cdcc4857b9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:36.586101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eab46c-670a-42a4-8b31-cdcc4857b9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:36.588238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eab46c-670a-42a4-8b31-cdcc4857b9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:36.605029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eab46c-670a-42a4-8b31-cdcc4857b9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:36.610667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eab46c-670a-42a4-8b31-cdcc4857b9b8 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:36.621441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1036/2067] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:36.685593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1032/2068] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:36 2022] PUT /identity/v3/projects/2a4953389e354347972e63683cb97a8a/users/b91c3e3a467b4f40a35551e2494c8180/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:36.685722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892821-03af-49ac-8b1a-bfca17bbe9cf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:36.685722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892821-03af-49ac-8b1a-bfca17bbe9cf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:36.685722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892821-03af-49ac-8b1a-bfca17bbe9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:36.739414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b5892821-03af-49ac-8b1a-bfca17bbe9cf None None] MFA Rules not processed for user `b91c3e3a467b4f40a35551e2494c8180`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:36.743349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b5892821-03af-49ac-8b1a-bfca17bbe9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:36.982543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:37.020328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1037/2069] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:37.051063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:37.051063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:37.290288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1345100-68fa-4b83-bb38-18f155edc55d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:37.297238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:37.298468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:37.299078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:37.299725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:37.319841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:37.322836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:37.335262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:37.346161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1033/2070] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:42.261170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0b1e-1aa9-4bcf-ab4d-f30cfd29ba47 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:42.262277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0b1e-1aa9-4bcf-ab4d-f30cfd29ba47 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:42.264354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0b1e-1aa9-4bcf-ab4d-f30cfd29ba47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:42.474618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-254f0b1e-1aa9-4bcf-ab4d-f30cfd29ba47 None None] MFA Rules not processed for user `043258ad0f2b4ec0a5217cee4429d68b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:42.476747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-254f0b1e-1aa9-4bcf-ab4d-f30cfd29ba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:42.848258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1038/2071] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 584 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:42.860409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:43.291301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2237d-40f9-48a3-95cd-4d5d723b14f9 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:43.310182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:43.311041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:43.311445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:43.311933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:43.315024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:43.319334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:43.518152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1034/2072] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:42 2022] POST /identity/v3/projects => generated 371 bytes in 658 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:43.524222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0b1e-1aa9-4bcf-ab4d-f30cfd29ba47 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:43.544576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0b1e-1aa9-4bcf-ab4d-f30cfd29ba47 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:43.665765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:43.666684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:43.667428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:43.669043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:43.669875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:43.680588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:43.830289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1039/2073] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:43 2022] POST /identity/v3/users => generated 292 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:43.834128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:43.966145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134c6fb-1e25-4ae0-aa76-47b2ae8b475b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:43.966145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:43.966145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:43.966145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:43.967211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:43.967211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:43.967211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:43.988234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1035/2074] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:43.989771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:44.012860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915099bc-b0a3-4a66-b9d0-cec29f88ca19 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:44.030080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:44.034808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:44.034808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:44.034808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] PATH_INFO: `/v3/projects/9bfa3a03b4544c61b36d2ccaa94e11e0/users/7e04675de8b040aabe7d758acd003201/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:44.081909 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bfa3a03b4544c61b36d2ccaa94e11e0, user_id=7e04675de8b040aabe7d758acd003201, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:44.104464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:44.260838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1040/2075] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:43 2022] PUT /identity/v3/projects/9bfa3a03b4544c61b36d2ccaa94e11e0/users/7e04675de8b040aabe7d758acd003201/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:44.264199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:44.277730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e39c6e-96f1-41ef-957e-2317e1e621d4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:44.482449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:44.483107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:44.483389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:44.488324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:44.489924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:44.492039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:44.526844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:44.547333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1036/2076] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:44.551351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cbd22-20a5-4507-80e1-a7fcc2cb6c32 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:44.564835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:44.576335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:44.577106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:44.577577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] PATH_INFO: `/v3/projects/9bfa3a03b4544c61b36d2ccaa94e11e0/users/7e04675de8b040aabe7d758acd003201/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:44.580553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bfa3a03b4544c61b36d2ccaa94e11e0, user_id=7e04675de8b040aabe7d758acd003201, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:44.583909 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:44.613065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1041/2077] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:44 2022] PUT /identity/v3/projects/9bfa3a03b4544c61b36d2ccaa94e11e0/users/7e04675de8b040aabe7d758acd003201/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:44.921693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:44.924035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:45.153552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4bccf-bc3c-4b36-b420-b7afdc869786 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:45.172705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:45.308698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3bb75-1d62-4bc7-be94-7fe9d6d32c35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:45.310401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3bb75-1d62-4bc7-be94-7fe9d6d32c35 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:45.311889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3bb75-1d62-4bc7-be94-7fe9d6d32c35 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:45.312181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3bb75-1d62-4bc7-be94-7fe9d6d32c35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:45.316147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f9a3bb75-1d62-4bc7-be94-7fe9d6d32c35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:45.338601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3bb75-1d62-4bc7-be94-7fe9d6d32c35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:45.340958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3bb75-1d62-4bc7-be94-7fe9d6d32c35 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:45.372220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1037/2078] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:45.433716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:45.628376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:45.741139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bdd6b-72d4-4dbb-abd9-d89b00ff2f1f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:45.762766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:45.765719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:45.768569 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:45.769124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] PATH_INFO: `/v3/users/52fa5939b312450daf12612d891cbc33` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:45.774341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] RBAC: Authorizing `identity:delete_user(user_id=52fa5939b312450daf12612d891cbc33)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:45.777488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:46.107045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52fa5939b312450daf12612d891cbc33'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:46.110897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52fa5939b312450daf12612d891cbc33'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:46.113492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52fa5939b312450daf12612d891cbc33'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:46.245133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52fa5939b312450daf12612d891cbc33'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:46.255481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52fa5939b312450daf12612d891cbc33'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:46.313869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1042/2079] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:45 2022] DELETE /identity/v3/users/52fa5939b312450daf12612d891cbc33 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:37:46.322043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:46.322963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:46.324379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:46.427871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:46.432955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:46.650202 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1038/2080] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:46.661777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:46.665972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:46.842566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b461b-bc3f-4f46-8c01-0679d03394da admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:46.854790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:46.855468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:46.855749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:46.856022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:46.883343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:46.889736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:46.913305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1043/2081] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:49.144626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:49.149072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:49.226316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33357b1-677a-427e-ac70-71b2f525a0ce None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:49.237165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:49.238575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:49.239003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:49.241490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] PATH_INFO: `/v3/projects/c5d89faecd2449c4b3b51fe739df15f1` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:49.243998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d89faecd2449c4b3b51fe739df15f1)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:49.248097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:49.642147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d89faecd2449c4b3b51fe739df15f1', 'request_id': 'req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:49.686543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d89faecd2449c4b3b51fe739df15f1', 'request_id': 'req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:49.856979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d89faecd2449c4b3b51fe739df15f1', 'request_id': 'req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:49.905686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] The token cache is being invalidate because project c5d89faecd2449c4b3b51fe739df15f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:49.907463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:49.908704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9adc2cca-cd5b-4b92-b6c6-7837eb88c915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:49.913875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1039/2082] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:49 2022] DELETE /identity/v3/projects/c5d89faecd2449c4b3b51fe739df15f1 => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:49.918614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:49.940353 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222781c3-287a-46c7-8477-0891090d26df service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:50.141337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:50.142045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:50.142323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:50.142597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] PATH_INFO: `/v3/users/b91c3e3a467b4f40a35551e2494c8180` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:50.143847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91c3e3a467b4f40a35551e2494c8180)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:50.146779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:50.150127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:50.274833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:50.274833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:50.274833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:50.470974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91c3e3a467b4f40a35551e2494c8180'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:50.472975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91c3e3a467b4f40a35551e2494c8180'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:50.474317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] MFA Rules not processed for user `7e04675de8b040aabe7d758acd003201`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:50.474893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91c3e3a467b4f40a35551e2494c8180'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:50.476510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:50.510260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:50.594979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91c3e3a467b4f40a35551e2494c8180'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:50.607657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91c3e3a467b4f40a35551e2494c8180'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:50.652702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1044/2083] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:37:49 2022] DELETE /identity/v3/users/b91c3e3a467b4f40a35551e2494c8180 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:50.666072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1040/2084] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:50.687162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:50.689388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:50.825465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ab80f-04a6-44c8-ac03-f6841ea70a1a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:50.833826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10babc-2e00-4e6d-bbe4-2023699e46ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:50.836102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10babc-2e00-4e6d-bbe4-2023699e46ca service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:50.836518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10babc-2e00-4e6d-bbe4-2023699e46ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:50.837118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10babc-2e00-4e6d-bbe4-2023699e46ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:50.839561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5a10babc-2e00-4e6d-bbe4-2023699e46ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:50.944658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10babc-2e00-4e6d-bbe4-2023699e46ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:50.946535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10babc-2e00-4e6d-bbe4-2023699e46ca service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:50.958893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1045/2085] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:37:51.586288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:51.588605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:51.638978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b743e6b-ef85-43af-a266-bc8fbe6b0298 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:51.784216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:51.785451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:51.786004 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:51.786847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] PATH_INFO: `/v3/projects/2a4953389e354347972e63683cb97a8a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:51.808289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4953389e354347972e63683cb97a8a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:51.811164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:52.090186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4953389e354347972e63683cb97a8a', 'request_id': 'req-80276c8f-cfdb-44ea-960d-ea1746f78afc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:52.101498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4953389e354347972e63683cb97a8a', 'request_id': 'req-80276c8f-cfdb-44ea-960d-ea1746f78afc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:52.253093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4953389e354347972e63683cb97a8a', 'request_id': 'req-80276c8f-cfdb-44ea-960d-ea1746f78afc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:52.290981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] The token cache is being invalidate because project 2a4953389e354347972e63683cb97a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:37:52.298597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:52.298597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-80276c8f-cfdb-44ea-960d-ea1746f78afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:37:52.299330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1041/2086] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:37:51 2022] DELETE /identity/v3/projects/2a4953389e354347972e63683cb97a8a => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:52.304339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaba0b-c059-4553-9beb-c9c295067f61 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:52.304941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaba0b-c059-4553-9beb-c9c295067f61 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:52.304941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaba0b-c059-4553-9beb-c9c295067f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:52.369619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ccfaba0b-c059-4553-9beb-c9c295067f61 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:52.373051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ccfaba0b-c059-4553-9beb-c9c295067f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:52.421053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:52.544147 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1046/2087] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:52.558147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2199a3-d4ec-443e-b83f-c59d8318bd7d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:52.558147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2199a3-d4ec-443e-b83f-c59d8318bd7d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:52.558569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2199a3-d4ec-443e-b83f-c59d8318bd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:52.604913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2f2199a3-d4ec-443e-b83f-c59d8318bd7d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:52.606774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2f2199a3-d4ec-443e-b83f-c59d8318bd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:52.708538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1042/2088] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:52.712087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaba0b-c059-4553-9beb-c9c295067f61 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:52.738837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaba0b-c059-4553-9beb-c9c295067f61 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:52.754413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a7b06-1e89-41e9-b04a-c0d59d3e6661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:52.756872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a7b06-1e89-41e9-b04a-c0d59d3e6661 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:52.757176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a7b06-1e89-41e9-b04a-c0d59d3e6661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:52.757434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a7b06-1e89-41e9-b04a-c0d59d3e6661 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:52.760352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a7b06-1e89-41e9-b04a-c0d59d3e6661 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:37:52.760740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a7b06-1e89-41e9-b04a-c0d59d3e6661 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:52.767595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a7b06-1e89-41e9-b04a-c0d59d3e6661 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:52.812762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1047/2089] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:37:52.820426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2199a3-d4ec-443e-b83f-c59d8318bd7d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:52.833121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2199a3-d4ec-443e-b83f-c59d8318bd7d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:52.835374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:52.836028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:52.836163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:52.860314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:52.861955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:52.867240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:52.869167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:52.869679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:52.870500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:52.871622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:52.874918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:53.047208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1048/2090] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:37:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:53.056050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:53.058764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:53.094520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1043/2091] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:52 2022] POST /identity/v3/projects => generated 385 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:53.099430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:53.168449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:53.339135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8651-9eaa-4cdf-93a6-762be89e01d3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:53.340586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b72dfe-995d-4b77-ab2f-4a0b515e8946 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:53.347747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:53.349389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:53.349860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:53.350321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:53.351291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:53.354513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:53.488824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:53.490198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:53.490814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:53.491934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:53.510121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:53.518209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1044/2092] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:53 2022] POST /identity/v3/users => generated 293 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:53.519656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:53.540260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ae639-2f43-473d-b9f8-9d8f6966f3bf admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:53.552366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:53.553362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:53.553828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:53.554434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:53.555286 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:53.557937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:53.576574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1045/2093] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:53.578044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:53.593540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc28707-0b88-4e08-9320-74f9cb004925 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:53.611621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:53.624782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:53.624782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:53.624782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:53.624782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:53.624782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:53.661801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1046/2094] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:53.665075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:53.666089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:53.667894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:53.682109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1049/2095] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:37:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:37:53.687544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a398506-6558-4f72-b8f7-d985e6a1e126 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:53.699098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7db3ce-19fe-46ef-9fdd-cc1b8cb229f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:53.700569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7db3ce-19fe-46ef-9fdd-cc1b8cb229f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:53.701354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7db3ce-19fe-46ef-9fdd-cc1b8cb229f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:53.702605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7db3ce-19fe-46ef-9fdd-cc1b8cb229f0 admin admin] PATH_INFO: `/v3/projects/cebed197072d4ed4ab75e5a00eea1c7a/users/b69fe258a8a54783a121e7cc3b02348a/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:53.786505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7db3ce-19fe-46ef-9fdd-cc1b8cb229f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebed197072d4ed4ab75e5a00eea1c7a, user_id=b69fe258a8a54783a121e7cc3b02348a, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:53.786505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7db3ce-19fe-46ef-9fdd-cc1b8cb229f0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:53.799033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:53.800590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:53.940406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce8f3d-4521-4625-9938-00a568f76b73 service swift] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:53.943859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1047/2096] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:53 2022] PUT /identity/v3/projects/cebed197072d4ed4ab75e5a00eea1c7a/users/b69fe258a8a54783a121e7cc3b02348a/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:53.970339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26cf7b-5d26-43d8-8f90-0f6a6115d6b0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:53.970671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26cf7b-5d26-43d8-8f90-0f6a6115d6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:53.971236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26cf7b-5d26-43d8-8f90-0f6a6115d6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:54.069775 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3c26cf7b-5d26-43d8-8f90-0f6a6115d6b0 None None] MFA Rules not processed for user `b69fe258a8a54783a121e7cc3b02348a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:54.073354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3c26cf7b-5d26-43d8-8f90-0f6a6115d6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:54.120736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:54.122096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:54.122518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:54.122854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:54.131099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:54.370664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1048/2097] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:37:54.372186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26cf7b-5d26-43d8-8f90-0f6a6115d6b0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:54.419920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:54.422256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:54.455763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1050/2098] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:37:54.604304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:54.637852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26cf7b-5d26-43d8-8f90-0f6a6115d6b0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:54.644528 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:54.646284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:54.647072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:54.647754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:54.648858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:54.652422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:54.814172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1049/2099] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:54 2022] POST /identity/v3/projects => generated 387 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:54.819156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:54.839949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def10cdc-55d2-4123-97e1-9b88b1146ac6 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:54.898941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:55.180986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:55.181762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:55.182041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:55.182431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:55.183184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:55.191801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:55.512528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1051/2100] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:54 2022] POST /identity/v3/users => generated 294 bytes in 694 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:55.523688 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:55.575222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d2569-4757-4f92-bbb6-2e8f27247c10 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:55.590040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:55.606050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:55.606050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:55.606050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:55.606050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:55.606050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:55.647562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1050/2101] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:55.651832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:55.664709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3289c5-1f3f-4a09-939d-1272c203b6d8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:55.671720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea2908-afec-46d2-af4e-3f04ff073c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:55.672550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea2908-afec-46d2-af4e-3f04ff073c5a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:55.673065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea2908-afec-46d2-af4e-3f04ff073c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:55.673420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea2908-afec-46d2-af4e-3f04ff073c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:55.674717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea2908-afec-46d2-af4e-3f04ff073c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:55.676427 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea2908-afec-46d2-af4e-3f04ff073c5a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:55.708668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1052/2102] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:55.713790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:55.730249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a771ce5-dcb2-40c6-80f4-707c219550b9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:55.742486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:55.744993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:55.745598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:55.746156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] PATH_INFO: `/v3/projects/f277c9bb2ac04be8a0d26f0eeef7df59/users/52f1626e73e64e2e98673abec93938c2/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:55.770748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f277c9bb2ac04be8a0d26f0eeef7df59, user_id=52f1626e73e64e2e98673abec93938c2, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:55.775279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:55.907006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1051/2103] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:55 2022] PUT /identity/v3/projects/f277c9bb2ac04be8a0d26f0eeef7df59/users/52f1626e73e64e2e98673abec93938c2/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:37:55.919012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9ba2-92dd-41d4-87d9-9b7501a21dc6 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:55.919886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9ba2-92dd-41d4-87d9-9b7501a21dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:55.920606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9ba2-92dd-41d4-87d9-9b7501a21dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:55.998677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-873a9ba2-92dd-41d4-87d9-9b7501a21dc6 None None] MFA Rules not processed for user `52f1626e73e64e2e98673abec93938c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:56.000124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-873a9ba2-92dd-41d4-87d9-9b7501a21dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:56.244088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1053/2104] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:56.247691 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:56.510653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5c083-3bad-4608-a7d7-ca8dbac56cd1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:56.523790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:56.527289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:56.527289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:56.527289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:56.527289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:56.530967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:56.700398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1052/2105] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:37:56 2022] POST /identity/v3/projects => generated 385 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:56.706118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a9ba2-92dd-41d4-87d9-9b7501a21dc6 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:56.734963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a9ba2-92dd-41d4-87d9-9b7501a21dc6 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:56.886398 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:56.887074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:56.887330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:56.887607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:56.888384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:56.891967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:57.047034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1054/2106] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:37:56 2022] POST /identity/v3/users => generated 299 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:37:57.052783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:57.072676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b846f9-3f01-417c-a0c1-6c5aa0b2f5e0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:57.081048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:57.082305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:57.086062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:57.086916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:57.088121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:57.091412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:57.120446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1053/2107] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:57.121701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:57.144029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c6017-6737-4c07-8857-eea0bda95b02 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:57.170042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:57.171561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:57.172145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:57.172422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] PATH_INFO: `/v3/projects/cda54e1e250d49f9ac5e0759914d09a2/users/993548c14afb46ffb48a8feb188134fb/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:57.197846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:57.205862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda54e1e250d49f9ac5e0759914d09a2, user_id=993548c14afb46ffb48a8feb188134fb, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:57.211099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:57.400932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1055/2108] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:57 2022] PUT /identity/v3/projects/cda54e1e250d49f9ac5e0759914d09a2/users/993548c14afb46ffb48a8feb188134fb/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:37:57.405642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:57.426678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08a2f1-3883-4e70-ad8b-27ac0bd7360f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:57.579550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16337f-db91-498f-8f2c-dd11df5b7718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:57.580217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16337f-db91-498f-8f2c-dd11df5b7718 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:57.580552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16337f-db91-498f-8f2c-dd11df5b7718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:57.581167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16337f-db91-498f-8f2c-dd11df5b7718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:57.581947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16337f-db91-498f-8f2c-dd11df5b7718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:57.585420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16337f-db91-498f-8f2c-dd11df5b7718 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:57.611545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:57.620121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1054/2109] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:37:57.626941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d0314-994a-4080-916f-a3608194b5c0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:57.644008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:57.644681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:57.644968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:57.645363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] PATH_INFO: `/v3/projects/cda54e1e250d49f9ac5e0759914d09a2/users/993548c14afb46ffb48a8feb188134fb/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:57.651793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda54e1e250d49f9ac5e0759914d09a2, user_id=993548c14afb46ffb48a8feb188134fb, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:57.657840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:57.751396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1056/2110] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:37:57 2022] PUT /identity/v3/projects/cda54e1e250d49f9ac5e0759914d09a2/users/993548c14afb46ffb48a8feb188134fb/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:37:57.772667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351bce-18e1-4d28-8948-65b8cb3fca85 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:57.773543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351bce-18e1-4d28-8948-65b8cb3fca85 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:57.773543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351bce-18e1-4d28-8948-65b8cb3fca85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:57.806932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:37:57.889418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d2351bce-18e1-4d28-8948-65b8cb3fca85 None None] MFA Rules not processed for user `993548c14afb46ffb48a8feb188134fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:37:57.890740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d2351bce-18e1-4d28-8948-65b8cb3fca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:58.240870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1055/2111] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:37:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:37:58.248016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:37:58.251169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:37:58.465533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0da6fd-851b-4133-a176-6a333e3faf00 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:37:58.469820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347264db-e1e1-4374-897e-6b1823c28f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:37:58.470528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347264db-e1e1-4374-897e-6b1823c28f3a service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:37:58.470829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347264db-e1e1-4374-897e-6b1823c28f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:37:58.471092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347264db-e1e1-4374-897e-6b1823c28f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:37:58.504752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347264db-e1e1-4374-897e-6b1823c28f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:37:58.509802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347264db-e1e1-4374-897e-6b1823c28f3a service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:37:58.548795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1057/2112] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:37:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:01.085759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2351bce-18e1-4d28-8948-65b8cb3fca85 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:01.118695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2351bce-18e1-4d28-8948-65b8cb3fca85 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:01.124829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:01.125843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:01.126359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:01.126829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:01.248079 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:01.250545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:01.268194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1056/2113] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:07.102707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab42e7a-7f4a-4fe1-b6f6-5471e2560728 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:07.103652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab42e7a-7f4a-4fe1-b6f6-5471e2560728 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:07.103652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab42e7a-7f4a-4fe1-b6f6-5471e2560728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:07.125499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7ab42e7a-7f4a-4fe1-b6f6-5471e2560728 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:07.135932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7ab42e7a-7f4a-4fe1-b6f6-5471e2560728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:07.193079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1058/2114] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 24 14:38:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:07.210687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:07.295865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0d4bb-9350-4740-aff7-cc78cd09a09b service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:07.301783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:07.309490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:07.309490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:07.309490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:07.313746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:07.315472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:07.327014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1057/2115] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:38:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:38:10.623947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab42e7a-7f4a-4fe1-b6f6-5471e2560728 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:10.635588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab42e7a-7f4a-4fe1-b6f6-5471e2560728 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:10.643023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:10.644133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:10.644755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:10.645731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:10.746650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:10.748648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:10.777763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1059/2116] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:12.476492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:12.479237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:12.528189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:12.581562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a1b78-fc38-4ab1-b51a-547d6f227444 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:12.587094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:12.587756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:12.588054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:12.588315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] PATH_INFO: `/v3/users/6e729065f1524fd9bf9ef3c6b42f7a5e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:12.589530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e729065f1524fd9bf9ef3c6b42f7a5e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:12.591578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:12.934248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e729065f1524fd9bf9ef3c6b42f7a5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:12.934954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e729065f1524fd9bf9ef3c6b42f7a5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:12.945442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e729065f1524fd9bf9ef3c6b42f7a5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:12.946645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e729065f1524fd9bf9ef3c6b42f7a5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:13.057585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9a9754b-132e-4160-b5c2-fa1a288ec185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e729065f1524fd9bf9ef3c6b42f7a5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:13.089533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1058/2117] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:12 2022] DELETE /identity/v3/users/6e729065f1524fd9bf9ef3c6b42f7a5e => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:13.628208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:13.649297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586d784-3377-475e-82b3-a66a1520efe1 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:13.697077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:13.764519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:13.765814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:13.766465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:13.768190 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] PATH_INFO: `/v3/projects/aa7908a26d224db1bf2e074c5f25b315` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:13.772245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7908a26d224db1bf2e074c5f25b315)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:13.775153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:14.026818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7908a26d224db1bf2e074c5f25b315', 'request_id': 'req-bba8ca2b-a89f-4830-b629-f5f75890729c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:14.035377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7908a26d224db1bf2e074c5f25b315', 'request_id': 'req-bba8ca2b-a89f-4830-b629-f5f75890729c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:14.045733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7908a26d224db1bf2e074c5f25b315', 'request_id': 'req-bba8ca2b-a89f-4830-b629-f5f75890729c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:14.246297 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] The token cache is being invalidate because project aa7908a26d224db1bf2e074c5f25b315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:14.246856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:14.247957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-bba8ca2b-a89f-4830-b629-f5f75890729c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:14.251526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1060/2118] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:13 2022] DELETE /identity/v3/projects/aa7908a26d224db1bf2e074c5f25b315 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:14.258103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9203b-2145-42ce-b242-84a425f5bf3c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:14.258996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9203b-2145-42ce-b242-84a425f5bf3c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:14.258996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9203b-2145-42ce-b242-84a425f5bf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:14.283315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-39c9203b-2145-42ce-b242-84a425f5bf3c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:14.284860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-39c9203b-2145-42ce-b242-84a425f5bf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:14.378185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1059/2119] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:14.388265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf6a1f-a53d-4317-91a1-acf254e0fb7c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:14.388569 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf6a1f-a53d-4317-91a1-acf254e0fb7c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:14.388838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf6a1f-a53d-4317-91a1-acf254e0fb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:14.418792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-99cf6a1f-a53d-4317-91a1-acf254e0fb7c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:14.420036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-99cf6a1f-a53d-4317-91a1-acf254e0fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:14.433925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1061/2120] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:14.439230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9203b-2145-42ce-b242-84a425f5bf3c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:14.450531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9203b-2145-42ce-b242-84a425f5bf3c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:14.465879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:14.467885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:14.468957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:14.470664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:14.472355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:38:14.476767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:14.482398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:14.501391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1060/2121] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:38:14.509114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf6a1f-a53d-4317-91a1-acf254e0fb7c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:14.523559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf6a1f-a53d-4317-91a1-acf254e0fb7c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:14.535551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:14.537023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:14.539073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:14.539073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:14.539073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:14.541305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:14.674186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1062/2122] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:38:14 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:38:14.679029 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:14.701166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d123e63-0a01-4a41-955d-e897543a573b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:14.757305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:14.830403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:14.831223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:14.831595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:14.831896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:14.834857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:14.837713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:15.075997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1061/2123] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:38:14 2022] POST /identity/v3/users => generated 293 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:15.087174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:15.108084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79a0f3-37cf-486f-8f17-a064723545c9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:15.115090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:15.116008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:15.116504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:15.117136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:15.117905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:15.121150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:15.150741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1063/2124] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:15.159111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:15.173409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970bbcb-2b34-4f97-ab6b-7c57b5a83304 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:15.187828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6bc7a-d67f-4d5e-9283-270f83773ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:15.188136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6bc7a-d67f-4d5e-9283-270f83773ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:15.188931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6bc7a-d67f-4d5e-9283-270f83773ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:15.188931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6bc7a-d67f-4d5e-9283-270f83773ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:15.189689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6bc7a-d67f-4d5e-9283-270f83773ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:15.191044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6bc7a-d67f-4d5e-9283-270f83773ca1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:15.219280 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1062/2125] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:15.221013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:15.243292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b14b3-4bce-4a11-8629-217e42ac66e5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:15.252800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:15.252800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:15.252800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:15.252800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] PATH_INFO: `/v3/projects/f6231f9c3c424e2599baac0368a2c23b/users/4362fe5b914047079c1d1141f5e83c4e/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:15.274081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6231f9c3c424e2599baac0368a2c23b, user_id=4362fe5b914047079c1d1141f5e83c4e, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:15.277379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:15.390399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1064/2126] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:15 2022] PUT /identity/v3/projects/f6231f9c3c424e2599baac0368a2c23b/users/4362fe5b914047079c1d1141f5e83c4e/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:15.407197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28946ad-606f-43fe-96bf-97ba82be0fde None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:15.407820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28946ad-606f-43fe-96bf-97ba82be0fde None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:15.408094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28946ad-606f-43fe-96bf-97ba82be0fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:15.450659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e28946ad-606f-43fe-96bf-97ba82be0fde None None] MFA Rules not processed for user `4362fe5b914047079c1d1141f5e83c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:15.452793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e28946ad-606f-43fe-96bf-97ba82be0fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:15.594122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1063/2127] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:15.713839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:15.715816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:15.790905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:15.801769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd1fd-70ab-4ed0-a01c-0beb9128e55c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:15.807856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d8f7c-6c05-4517-b909-f69a2e88eee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:15.809071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8f7c-6c05-4517-b909-f69a2e88eee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:15.809360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8f7c-6c05-4517-b909-f69a2e88eee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:15.809635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d8f7c-6c05-4517-b909-f69a2e88eee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:15.821743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d8f7c-6c05-4517-b909-f69a2e88eee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:15.823451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d8f7c-6c05-4517-b909-f69a2e88eee0 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:15.835315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1065/2128] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:38:16.254191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1128-16dd-44c8-9b91-07feae6ef24d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:16.255278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1128-16dd-44c8-9b91-07feae6ef24d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:16.255751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1128-16dd-44c8-9b91-07feae6ef24d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:16.258338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1064/2129] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:38:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:38:16.264844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:16.265159 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:16.265504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:16.288300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:16.289831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:16.372232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1066/2130] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:38:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:16.388361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a1128-16dd-44c8-9b91-07feae6ef24d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:16.391020 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5c4a1128-16dd-44c8-9b91-07feae6ef24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:16.469182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a1128-16dd-44c8-9b91-07feae6ef24d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:16.475404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3759bd7-978f-4afd-969e-8dccd85aad13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:16.477043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3759bd7-978f-4afd-969e-8dccd85aad13 service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:16.477608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3759bd7-978f-4afd-969e-8dccd85aad13 service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:16.477962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3759bd7-978f-4afd-969e-8dccd85aad13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:16.502306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3759bd7-978f-4afd-969e-8dccd85aad13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:16.504121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3759bd7-978f-4afd-969e-8dccd85aad13 service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:16.518420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1065/2131] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:38:16.878650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:16.881291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:16.889300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd90dd-6b13-40eb-b479-c139d28e0381 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:16.889847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd90dd-6b13-40eb-b479-c139d28e0381 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:16.890416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd90dd-6b13-40eb-b479-c139d28e0381 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:16.892430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1066/2132] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 14:38:16.907463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:16.908994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:16.910187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:16.956621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] MFA Rules not processed for user `48364ccf19434c4c9c09b4e5ce4bfc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:16.959425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:17.046531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:17.096888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1067/2133] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:38:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:17.098518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17394c-899d-47cf-8101-180eab7d4cc8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:17.108842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:17.110126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:17.110625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:17.111081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] PATH_INFO: `/v3/users/b69fe258a8a54783a121e7cc3b02348a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:17.113765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69fe258a8a54783a121e7cc3b02348a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:17.118244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:17.146775 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:17.149271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:17.177566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbdcca-9da6-4762-b764-fc87ef2bd9ba None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:17.185245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874fd9f1-5cd4-44e5-907b-f65e220ab918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:17.187199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fd9f1-5cd4-44e5-907b-f65e220ab918 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:17.187199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fd9f1-5cd4-44e5-907b-f65e220ab918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:17.187534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fd9f1-5cd4-44e5-907b-f65e220ab918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:17.196764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874fd9f1-5cd4-44e5-907b-f65e220ab918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:17.198530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874fd9f1-5cd4-44e5-907b-f65e220ab918 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:17.211072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1068/2134] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:17.336910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478108de-281c-4090-b82b-31acf234b503 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:17.336910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478108de-281c-4090-b82b-31acf234b503 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:17.336910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478108de-281c-4090-b82b-31acf234b503 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:17.336910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1069/2135] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:38:17.344694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:17.344996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:17.345374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:17.376700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:17.381359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:17.396863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1070/2136] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:38:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:38:17.420562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:17.424555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:17.435682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69fe258a8a54783a121e7cc3b02348a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:17.439453 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69fe258a8a54783a121e7cc3b02348a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:17.439453 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69fe258a8a54783a121e7cc3b02348a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:17.446138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ced01-bad7-4e7d-95f3-b9fa614cbe54 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:17.450964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd4832-0956-4eb0-91b1-61a563b34eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:17.451765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4832-0956-4eb0-91b1-61a563b34eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:17.452326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4832-0956-4eb0-91b1-61a563b34eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:17.452668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4832-0956-4eb0-91b1-61a563b34eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:17.455249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bcfd4832-0956-4eb0-91b1-61a563b34eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:17.482268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd4832-0956-4eb0-91b1-61a563b34eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:17.485256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd4832-0956-4eb0-91b1-61a563b34eb8 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:17.498323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1071/2137] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:17.576320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69fe258a8a54783a121e7cc3b02348a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:17.585910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69fe258a8a54783a121e7cc3b02348a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:17.612726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1067/2138] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:16 2022] DELETE /identity/v3/users/b69fe258a8a54783a121e7cc3b02348a => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:17.618805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:17.619248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:17.619982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:17.641373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:17.642890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:17.773344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1072/2139] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:17.787263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:17.789061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:17.807931 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:17.925519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825af5-99eb-4bf2-abfe-818bbdf9974e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:17.931874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:17.932779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:17.933212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:17.933699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:17.949972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:17.951719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:17.970595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1068/2140] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:19.112900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:19.116511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:19.131100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246864c5-3a16-4cc1-953c-7b91d16091af None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:19.137489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:19.138260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:19.138539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:19.138809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] PATH_INFO: `/v3/projects/cebed197072d4ed4ab75e5a00eea1c7a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:19.140229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebed197072d4ed4ab75e5a00eea1c7a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:19.142383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:19.331777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebed197072d4ed4ab75e5a00eea1c7a', 'request_id': 'req-cf297354-aa71-489b-afa0-6416480e84b9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:19.343651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebed197072d4ed4ab75e5a00eea1c7a', 'request_id': 'req-cf297354-aa71-489b-afa0-6416480e84b9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:19.476978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebed197072d4ed4ab75e5a00eea1c7a', 'request_id': 'req-cf297354-aa71-489b-afa0-6416480e84b9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:19.501316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] The token cache is being invalidate because project cebed197072d4ed4ab75e5a00eea1c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:19.504141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:19.504141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:19.506610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1073/2141] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:19 2022] DELETE /identity/v3/projects/cebed197072d4ed4ab75e5a00eea1c7a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:38:19.512633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:19.523939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62244377-64df-4cd2-8a2a-7389fba9351d service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:19.610034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:19.610900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:19.611217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:19.611498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] PATH_INFO: `/v3/users/52f1626e73e64e2e98673abec93938c2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:19.612913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f1626e73e64e2e98673abec93938c2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:19.616219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:19.738279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:19.742052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:19.770380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297354-aa71-489b-afa0-6416480e84b9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:19.779101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:19.780119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:19.780589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:19.781013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] PATH_INFO: `/v3/users/4362fe5b914047079c1d1141f5e83c4e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:19.783424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4362fe5b914047079c1d1141f5e83c4e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:19.787740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:19.927732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f1626e73e64e2e98673abec93938c2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:19.928485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f1626e73e64e2e98673abec93938c2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:19.929165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f1626e73e64e2e98673abec93938c2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.062894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f1626e73e64e2e98673abec93938c2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.081757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f1626e73e64e2e98673abec93938c2'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.096734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:20.130376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4362fe5b914047079c1d1141f5e83c4e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.132777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4362fe5b914047079c1d1141f5e83c4e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.149359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4362fe5b914047079c1d1141f5e83c4e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.151639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4362fe5b914047079c1d1141f5e83c4e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.153688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1069/2142] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:19 2022] DELETE /identity/v3/users/52f1626e73e64e2e98673abec93938c2 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:38:20.155334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:20.167696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:20.246720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:20.247035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d46a4bb2-282d-4975-99be-0c0c2621ddab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4362fe5b914047079c1d1141f5e83c4e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.282117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1074/2143] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:19 2022] DELETE /identity/v3/users/4362fe5b914047079c1d1141f5e83c4e => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:38:20.289766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:20.290130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:20.290500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:20.324832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:20.326235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:20.336564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb21192-0d05-40e2-8bd2-53c8f79cfe5a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:20.431669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1075/2144] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:20.467791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:20.483188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:20.532639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:20.534823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:20.537230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:20.537806 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] PATH_INFO: `/v3/users/043258ad0f2b4ec0a5217cee4429d68b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:20.539846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=043258ad0f2b4ec0a5217cee4429d68b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:20.543386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:20.620641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c55ef-9072-4d45-8a34-8f0ec6f2fabb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:20.624831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:20.625477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:20.625774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:20.626082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:20.637163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:20.638665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:20.653439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1076/2145] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:20.879831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043258ad0f2b4ec0a5217cee4429d68b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.881976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043258ad0f2b4ec0a5217cee4429d68b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.889355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043258ad0f2b4ec0a5217cee4429d68b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.987798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043258ad0f2b4ec0a5217cee4429d68b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:20.998061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043258ad0f2b4ec0a5217cee4429d68b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.024168 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1070/2146] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:20 2022] DELETE /identity/v3/users/043258ad0f2b4ec0a5217cee4429d68b => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:21.107577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:21.116194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:21.220421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceae013-8963-4830-b4f6-4b593a989203 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:21.444298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:21.446347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:21.498062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:21.523744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:21.524747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:21.527987 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:21.528507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] PATH_INFO: `/v3/projects/f277c9bb2ac04be8a0d26f0eeef7df59` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:21.534099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f277c9bb2ac04be8a0d26f0eeef7df59)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:21.536972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:21.619991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:21.667451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286f0d7-5bbd-4165-b2f2-93946676b884 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:21.687336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:21.693026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:21.693026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:21.693026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] PATH_INFO: `/v3/projects/d0d567af87e941398ce937433dae18c7` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:21.693026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0d567af87e941398ce937433dae18c7)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:21.694947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:21.763937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f277c9bb2ac04be8a0d26f0eeef7df59', 'request_id': 'req-85609639-f0c0-4e2e-a640-7b93e168b923'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.774513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f277c9bb2ac04be8a0d26f0eeef7df59', 'request_id': 'req-85609639-f0c0-4e2e-a640-7b93e168b923'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.869805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f277c9bb2ac04be8a0d26f0eeef7df59', 'request_id': 'req-85609639-f0c0-4e2e-a640-7b93e168b923'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.870880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d567af87e941398ce937433dae18c7', 'request_id': 'req-f951e1c8-484b-4279-b1f5-943a4379cc14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.890848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d567af87e941398ce937433dae18c7', 'request_id': 'req-f951e1c8-484b-4279-b1f5-943a4379cc14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.896585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] The token cache is being invalidate because project f277c9bb2ac04be8a0d26f0eeef7df59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:21.896876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.897888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:21.899851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1077/2147] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:21 2022] DELETE /identity/v3/projects/f277c9bb2ac04be8a0d26f0eeef7df59 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:21.904971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:21.906627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:21.911180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d567af87e941398ce937433dae18c7', 'request_id': 'req-f951e1c8-484b-4279-b1f5-943a4379cc14'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.024591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] The token cache is being invalidate because project d0d567af87e941398ce937433dae18c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:22.026385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.026385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.027418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1071/2148] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:21 2022] DELETE /identity/v3/projects/d0d567af87e941398ce937433dae18c7 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:38:22.035352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:22.040165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:22.093824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609639-f0c0-4e2e-a640-7b93e168b923 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:22.283294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951e1c8-484b-4279-b1f5-943a4379cc14 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:22.297345 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:22.298258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:22.298707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:22.300309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] PATH_INFO: `/v3/users/7e04675de8b040aabe7d758acd003201` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:22.306928 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:22.307804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:22.307950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:22.308254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] PATH_INFO: `/v3/users/993548c14afb46ffb48a8feb188134fb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:22.310701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993548c14afb46ffb48a8feb188134fb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:22.311315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e04675de8b040aabe7d758acd003201)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:22.313502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:22.319428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:22.713344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e04675de8b040aabe7d758acd003201'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.714738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e04675de8b040aabe7d758acd003201'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.716530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e04675de8b040aabe7d758acd003201'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.718998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993548c14afb46ffb48a8feb188134fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.720208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993548c14afb46ffb48a8feb188134fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.735237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993548c14afb46ffb48a8feb188134fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.736011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993548c14afb46ffb48a8feb188134fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.822364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e04675de8b040aabe7d758acd003201'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.841402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e04675de8b040aabe7d758acd003201'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.867072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1072/2149] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:22 2022] DELETE /identity/v3/users/7e04675de8b040aabe7d758acd003201 => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:22.869179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:22.871888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:22.910594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24104907-f374-4bd3-b376-76fdc5360f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993548c14afb46ffb48a8feb188134fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:22.974226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1078/2150] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:21 2022] DELETE /identity/v3/users/993548c14afb46ffb48a8feb188134fb => generated 0 bytes in 1071 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:23.007094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea95a06-a658-4fe8-8a0e-9528a7974ff7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:23.089461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:23.090870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.091239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.091607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] PATH_INFO: `/v3/projects/f6231f9c3c424e2599baac0368a2c23b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.094253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6231f9c3c424e2599baac0368a2c23b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:23.097501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:23.142661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:23.251631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:23.308094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6231f9c3c424e2599baac0368a2c23b', 'request_id': 'req-aa17e2da-6a93-4d13-a36c-3160779c35b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.319910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6231f9c3c424e2599baac0368a2c23b', 'request_id': 'req-aa17e2da-6a93-4d13-a36c-3160779c35b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.334864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6231f9c3c424e2599baac0368a2c23b', 'request_id': 'req-aa17e2da-6a93-4d13-a36c-3160779c35b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.429727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] The token cache is being invalidate because project f6231f9c3c424e2599baac0368a2c23b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:23.429937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.430697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.432536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1073/2151] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:22 2022] DELETE /identity/v3/projects/f6231f9c3c424e2599baac0368a2c23b => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:23.438194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c787c-e985-4a7c-852e-201c91e120e4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.439981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c787c-e985-4a7c-852e-201c91e120e4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.440495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c787c-e985-4a7c-852e-201c91e120e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.467082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:23.468939 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:23.532394 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1c3c787c-e985-4a7c-852e-201c91e120e4 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:23.538071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1c3c787c-e985-4a7c-852e-201c91e120e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:23.586572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e2da-6a93-4d13-a36c-3160779c35b6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:23.593670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:23.595365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.595604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.595970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] PATH_INFO: `/v3/projects/9bfa3a03b4544c61b36d2ccaa94e11e0` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.598838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bfa3a03b4544c61b36d2ccaa94e11e0)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:23.600979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:23.641764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1079/2152] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:23.651963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decffb7-9628-4f37-9c33-20cc7e17043a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.652252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decffb7-9628-4f37-9c33-20cc7e17043a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.652530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decffb7-9628-4f37-9c33-20cc7e17043a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.676508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3decffb7-9628-4f37-9c33-20cc7e17043a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:23.678571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3decffb7-9628-4f37-9c33-20cc7e17043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:23.689620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1080/2153] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:38:23.695493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decffb7-9628-4f37-9c33-20cc7e17043a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:23.708209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decffb7-9628-4f37-9c33-20cc7e17043a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:23.714375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:23.715009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.715395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.715553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.716226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:38:23.716536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:23.718264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:23.737278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bfa3a03b4544c61b36d2ccaa94e11e0', 'request_id': 'req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.737876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1081/2154] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:38:23.744487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bfa3a03b4544c61b36d2ccaa94e11e0', 'request_id': 'req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.752063 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bfa3a03b4544c61b36d2ccaa94e11e0', 'request_id': 'req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.755267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:23.770939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9af1c-708d-40a5-b2fa-bffd8ffa0e86 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:23.846292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85b270-d00b-408f-9647-34d9b26c3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:23.846934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85b270-d00b-408f-9647-34d9b26c3fed admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.847214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85b270-d00b-408f-9647-34d9b26c3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.847488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85b270-d00b-408f-9647-34d9b26c3fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.848196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85b270-d00b-408f-9647-34d9b26c3fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:23.851030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85b270-d00b-408f-9647-34d9b26c3fed admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:23.879759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] The token cache is being invalidate because project 9bfa3a03b4544c61b36d2ccaa94e11e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:23.879759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.879759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b2cb8bfd-30c2-4acc-aa19-fdb2ed92ccc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:23.879759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1074/2155] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:23 2022] DELETE /identity/v3/projects/9bfa3a03b4544c61b36d2ccaa94e11e0 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:23.879759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.881058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.881058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.896426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:23.897371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:23.958769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1075/2156] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:38:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:23.959815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:23.961175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:23.975946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ce35-28c5-43ad-8f8e-5e3a9df8344f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:23.980705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:23.981357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:23.981643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:23.981979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] PATH_INFO: `/v3/projects/cda54e1e250d49f9ac5e0759914d09a2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:23.983210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda54e1e250d49f9ac5e0759914d09a2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:23.985137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.018690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1082/2157] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:38:23 2022] POST /identity/v3/projects => generated 365 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:24.020918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.021200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.021539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.044871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:24.046400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:24.135181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1083/2158] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:24.135405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda54e1e250d49f9ac5e0759914d09a2', 'request_id': 'req-f939a322-5325-461c-964b-31ea021862d3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:24.135509 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.135509 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:24.138074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda54e1e250d49f9ac5e0759914d09a2', 'request_id': 'req-f939a322-5325-461c-964b-31ea021862d3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:24.146706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda54e1e250d49f9ac5e0759914d09a2', 'request_id': 'req-f939a322-5325-461c-964b-31ea021862d3'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:24.228891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484e06-c6fa-4547-90ae-6eaa24463f37 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.266623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] The token cache is being invalidate because project cda54e1e250d49f9ac5e0759914d09a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:24.267192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:24.268216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:24.271386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1076/2159] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:23 2022] DELETE /identity/v3/projects/cda54e1e250d49f9ac5e0759914d09a2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:24.278031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.279858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:24.296743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:24.329490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:24.335296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bbb45-97ab-4ead-afc7-d4836aedf46a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.336194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bbb45-97ab-4ead-afc7-d4836aedf46a service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.336602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bbb45-97ab-4ead-afc7-d4836aedf46a service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.336938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bbb45-97ab-4ead-afc7-d4836aedf46a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.339410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-806bbb45-97ab-4ead-afc7-d4836aedf46a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:24.410302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939a322-5325-461c-964b-31ea021862d3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.419093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7b7e4-7d26-4070-a8f4-ea69980a258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.420299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7b7e4-7d26-4070-a8f4-ea69980a258d admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.421220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7b7e4-7d26-4070-a8f4-ea69980a258d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.421731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7b7e4-7d26-4070-a8f4-ea69980a258d admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.423042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7b7e4-7d26-4070-a8f4-ea69980a258d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.427304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7b7e4-7d26-4070-a8f4-ea69980a258d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.459742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bbb45-97ab-4ead-afc7-d4836aedf46a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.461856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bbb45-97ab-4ead-afc7-d4836aedf46a service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.472604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1084/2160] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:38:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:38:24.475764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9279-65e8-4830-8788-f8ea4487fb48 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.476230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9279-65e8-4830-8788-f8ea4487fb48 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.476602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9279-65e8-4830-8788-f8ea4487fb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.517386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5e7b9279-65e8-4830-8788-f8ea4487fb48 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:24.518857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5e7b9279-65e8-4830-8788-f8ea4487fb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:24.533168 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1077/2161] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:38:24 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:24.539220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56b20-9c0e-4562-a736-87ff9850010f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.539220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56b20-9c0e-4562-a736-87ff9850010f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.539674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1085/2162] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:24.539819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a56b20-9c0e-4562-a736-87ff9850010f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.539953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b9279-65e8-4830-8788-f8ea4487fb48 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.558428 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b9279-65e8-4830-8788-f8ea4487fb48 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.560133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-73a56b20-9c0e-4562-a736-87ff9850010f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:24.561419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-73a56b20-9c0e-4562-a736-87ff9850010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:24.567623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e58fb-2170-4aa2-9854-78e9ae9795e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.568571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e58fb-2170-4aa2-9854-78e9ae9795e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.569085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e58fb-2170-4aa2-9854-78e9ae9795e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.569562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e58fb-2170-4aa2-9854-78e9ae9795e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.570709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e58fb-2170-4aa2-9854-78e9ae9795e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.572820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1078/2163] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:24.573502 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56b20-9c0e-4562-a736-87ff9850010f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.579315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e58fb-2170-4aa2-9854-78e9ae9795e8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.585160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a56b20-9c0e-4562-a736-87ff9850010f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.591230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.591855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.592135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.592408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.593081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:38:24.593390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.595129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.609277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1086/2164] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:38:24.609277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b4ce-79e2-48a9-84d9-2a81c6b6475f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.609277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b4ce-79e2-48a9-84d9-2a81c6b6475f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.609807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b4ce-79e2-48a9-84d9-2a81c6b6475f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.622802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1079/2165] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:38:24.624119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.641856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-38b8b4ce-79e2-48a9-84d9-2a81c6b6475f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:24.642981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b371-f94b-4372-95d5-0184014197e5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.643703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-38b8b4ce-79e2-48a9-84d9-2a81c6b6475f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:24.651200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.652418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.652572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.652927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.653893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.654736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1087/2166] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:24.656032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b4ce-79e2-48a9-84d9-2a81c6b6475f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.657004 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.680875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1080/2167] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:24.680875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.690376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b4ce-79e2-48a9-84d9-2a81c6b6475f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.696026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d407d0-9e04-4e62-bbef-781445dcb261 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.701458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.703153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.703153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.703520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.704667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.709371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.712924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.713579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.713856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.728273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.728273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:38:24.728273 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.728842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:24.755230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1081/2168] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:38:24.755230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.787805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee6dc6-38de-4def-aa3a-46a6f2a91efe admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:24.809764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:24.811094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:24.811697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:24.814027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] PATH_INFO: `/v3/projects/2ed0a2d1734f414c8e34f24e8644a0e6/users/06c17524efb04a9e8ee52a2c388adc5f/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:24.846466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:24.856518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1088/2169] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:38:24 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:24.860204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:24.920371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed0a2d1734f414c8e34f24e8644a0e6, user_id=06c17524efb04a9e8ee52a2c388adc5f, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:24.935965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:24.938276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.011752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad874-dba0-4806-b86c-642573be3d0f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.018246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f189e-75a2-458e-959c-863a46fe75d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.025236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f189e-75a2-458e-959c-863a46fe75d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.025236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f189e-75a2-458e-959c-863a46fe75d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.025236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f189e-75a2-458e-959c-863a46fe75d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.025236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f189e-75a2-458e-959c-863a46fe75d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:25.025236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f189e-75a2-458e-959c-863a46fe75d2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.094186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1082/2170] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:24 2022] PUT /identity/v3/projects/2ed0a2d1734f414c8e34f24e8644a0e6/users/06c17524efb04a9e8ee52a2c388adc5f/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:25.094915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.249596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0aa43-ae9c-4de0-a741-648bf535a401 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.264390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.264390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.264390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.264390 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.265035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:25.265192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1089/2171] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:38:24 2022] POST /identity/v3/projects => generated 393 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:25.265339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.269260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429012c-5455-48c5-9c90-abaa17441530 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.269714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429012c-5455-48c5-9c90-abaa17441530 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.270383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429012c-5455-48c5-9c90-abaa17441530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.320084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0429012c-5455-48c5-9c90-abaa17441530 None None] MFA Rules not processed for user `06c17524efb04a9e8ee52a2c388adc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:25.321970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0429012c-5455-48c5-9c90-abaa17441530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:25.368457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1083/2172] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:38:25 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:25.373795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.409146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27e51-2686-4664-b3de-fad2e296b353 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.428938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:25.502031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1090/2173] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:25.505145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429012c-5455-48c5-9c90-abaa17441530 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.575091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:25.592731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.593596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.594130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.594459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.595157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:25.598534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.671629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429012c-5455-48c5-9c90-abaa17441530 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.687762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.688514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.693327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.693327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.693327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:25.693327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.722885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1091/2174] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:25.727730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.728432 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:25.762422 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1084/2175] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:38:25 2022] POST /identity/v3/users => generated 297 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:25.765064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.780280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ff98-b547-4e53-b5ca-63b9f854cedc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.789038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.791609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.792487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.793047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.794489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:25.798012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.830729 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1085/2176] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:25.832132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.862642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93b47b-4778-4688-a7d2-a5ebb2f45665 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.876926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.878593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.880235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64feb-2d27-45d6-8c01-216594893605 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.881473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.882469 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.885242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:25.888681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.893262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.894451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.895063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.895623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.920708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1086/2177] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:25.920708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.929667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:25.932769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:25.957079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b084154-d35c-46bd-8ee8-e92c8a56e26f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:25.961758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1092/2178] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:25.964136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:25.979856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ced82-b800-466f-94a3-4d26381bc8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:25.981034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ced82-b800-466f-94a3-4d26381bc8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:25.981444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ced82-b800-466f-94a3-4d26381bc8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:25.982227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ced82-b800-466f-94a3-4d26381bc8fe admin admin] PATH_INFO: `/v3/projects/c28c72c2473844649fe8ac3c8245ac80/users/831139f58d58411cb73cb8076f2980da/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:25.987938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41cf56-564a-4aa0-b06e-ba3096332b5d service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:26.017242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:26.019526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:26.020302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:26.021336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:26.022940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:26.026915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:26.074024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ced82-b800-466f-94a3-4d26381bc8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28c72c2473844649fe8ac3c8245ac80, user_id=831139f58d58411cb73cb8076f2980da, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:26.081140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1093/2179] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:26.087678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ced82-b800-466f-94a3-4d26381bc8fe admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:26.088374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:26.114084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd60cd-afee-4683-88a8-c8f20fc38ee2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:26.123958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3ae43-606a-4925-83f3-a07eef008d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:26.125290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ae43-606a-4925-83f3-a07eef008d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:26.126038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ae43-606a-4925-83f3-a07eef008d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:26.126588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ae43-606a-4925-83f3-a07eef008d28 admin admin] PATH_INFO: `/v3/projects/47b252dbc3d94a91852d797c731c2e97/users/231d9da5dd5a43008e2adb89c3ee675b/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:26.163349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3ae43-606a-4925-83f3-a07eef008d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b252dbc3d94a91852d797c731c2e97, user_id=231d9da5dd5a43008e2adb89c3ee675b, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:26.169667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3ae43-606a-4925-83f3-a07eef008d28 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:26.389631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1087/2180] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:25 2022] PUT /identity/v3/projects/c28c72c2473844649fe8ac3c8245ac80/users/831139f58d58411cb73cb8076f2980da/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:26.409381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:26.411794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:26.411794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:26.435620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1094/2181] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:26 2022] PUT /identity/v3/projects/47b252dbc3d94a91852d797c731c2e97/users/231d9da5dd5a43008e2adb89c3ee675b/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:38:26.449875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5a869-c298-4007-afe8-f139cfc2584f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:26.452755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5a869-c298-4007-afe8-f139cfc2584f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:26.454735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5a869-c298-4007-afe8-f139cfc2584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:26.509814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] MFA Rules not processed for user `831139f58d58411cb73cb8076f2980da`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:26.512787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:26.516435 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3bb5a869-c298-4007-afe8-f139cfc2584f None None] MFA Rules not processed for user `231d9da5dd5a43008e2adb89c3ee675b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:26.520359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3bb5a869-c298-4007-afe8-f139cfc2584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:26.681339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1095/2182] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:26.687710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1088/2183] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:26.690228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5a869-c298-4007-afe8-f139cfc2584f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:26.708550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:26.720388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:26.759917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:26.861823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5a869-c298-4007-afe8-f139cfc2584f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:26.875271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:26.876540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:26.877011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:26.877729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:26.880866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:26.885770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:26.938890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:26.962642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d956c-7e9c-4c6e-8b57-17b17e02fdfd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:26.967023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:26.967753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:26.968112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:26.968459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:26.980725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:26.982438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:27.001494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1089/2184] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:27.046411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1096/2185] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:38:26 2022] POST /identity/v3/projects => generated 393 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:27.047256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:27.060305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb97c8-bd3a-4045-925e-cd23a637abea service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:27.163941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:27.164632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:27.164919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:27.165196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:27.166366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:27.169100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:27.376316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1090/2186] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:38:27 2022] POST /identity/v3/users => generated 303 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:27.382164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:27.396354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0453b-a0d0-48cf-b71e-8e7d97d50d75 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:27.402812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:27.403454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:27.403727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:27.403998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:27.404660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:27.406523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:27.434415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1097/2187] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:38:27.438176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:27.452801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558fb32-8bc7-406c-bf52-62a8a154ff08 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:27.459688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:27.460819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:27.461201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:27.461659 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] PATH_INFO: `/v3/projects/cb85eff649f74198830f70fc226c5ac7/users/31d60812b7f34271ae02396005c49652/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:27.485262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb85eff649f74198830f70fc226c5ac7, user_id=31d60812b7f34271ae02396005c49652, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:27.489123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:27.656877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1091/2188] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:27 2022] PUT /identity/v3/projects/cb85eff649f74198830f70fc226c5ac7/users/31d60812b7f34271ae02396005c49652/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:27.661151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:27.672614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a66b3-ba87-4950-be0d-9f619a49da12 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:27.811634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372a399-0ea3-41fe-b1bd-36d3f4410e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:27.813032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372a399-0ea3-41fe-b1bd-36d3f4410e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:27.813716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372a399-0ea3-41fe-b1bd-36d3f4410e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:27.814209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372a399-0ea3-41fe-b1bd-36d3f4410e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:27.815364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372a399-0ea3-41fe-b1bd-36d3f4410e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:27.819035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372a399-0ea3-41fe-b1bd-36d3f4410e5e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:27.836169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1098/2189] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:27.843082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:27.881870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb95b8c-41d4-41ff-8ced-090e0e99d2e2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:27.895441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:27.896664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:27.897121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:27.898557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] PATH_INFO: `/v3/projects/cb85eff649f74198830f70fc226c5ac7/users/31d60812b7f34271ae02396005c49652/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:27.903728 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb85eff649f74198830f70fc226c5ac7, user_id=31d60812b7f34271ae02396005c49652, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:27.914799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:27.935097 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1092/2190] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:27 2022] PUT /identity/v3/projects/cb85eff649f74198830f70fc226c5ac7/users/31d60812b7f34271ae02396005c49652/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:27.958261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312259-bdfa-4e30-9630-ec593d58212c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:27.958715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312259-bdfa-4e30-9630-ec593d58212c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:27.959168 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312259-bdfa-4e30-9630-ec593d58212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:28.056974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-af312259-bdfa-4e30-9630-ec593d58212c None None] MFA Rules not processed for user `31d60812b7f34271ae02396005c49652`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:28.059279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-af312259-bdfa-4e30-9630-ec593d58212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:28.212962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1099/2191] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:28.401867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:28.617603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5536e-10a3-4467-a522-6882e315ed15 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:28.617603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:28.617603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:28.617603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:28.621077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:28.621077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:28.621077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:28.621077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1093/2192] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:28.685686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af312259-bdfa-4e30-9630-ec593d58212c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:28.699562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af312259-bdfa-4e30-9630-ec593d58212c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:28.709870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67f28b-07ce-4da6-8e6f-fad81648adbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:28.710565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67f28b-07ce-4da6-8e6f-fad81648adbd service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:28.710876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67f28b-07ce-4da6-8e6f-fad81648adbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:28.711162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67f28b-07ce-4da6-8e6f-fad81648adbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:28.755373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:28.887838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67f28b-07ce-4da6-8e6f-fad81648adbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:28.891479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67f28b-07ce-4da6-8e6f-fad81648adbd service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:28.910022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1100/2193] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:38.558934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:38.795385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:38.928393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93936e5a-9933-4cae-8947-088025a93ea7 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:38.945344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:38.945344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:38.945344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:38.945344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] PATH_INFO: `/v3/users/231d9da5dd5a43008e2adb89c3ee675b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:38.945896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=231d9da5dd5a43008e2adb89c3ee675b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:38.948277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:39.266243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231d9da5dd5a43008e2adb89c3ee675b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:39.267346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231d9da5dd5a43008e2adb89c3ee675b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:39.268308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231d9da5dd5a43008e2adb89c3ee675b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:39.376221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231d9da5dd5a43008e2adb89c3ee675b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:39.387315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231d9da5dd5a43008e2adb89c3ee675b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:39.413314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1094/2194] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:38 2022] DELETE /identity/v3/users/231d9da5dd5a43008e2adb89c3ee675b => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:39.419907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:39.420880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:39.420880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:39.446963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:39.446963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:39.524258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1101/2195] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:38:39.534850 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:39.537354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:39.608726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962d9b-bb4b-4776-a16b-66b2252cba63 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:39.613238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:39.614748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:39.615066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:39.615337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:39.632111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:39.635287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:39.655306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1095/2196] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:38:40.556151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:40.560075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:40.583038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afc231-72cd-4381-aa81-0c6efafef041 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:40.590229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:40.591507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:40.592371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:40.592931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] PATH_INFO: `/v3/projects/47b252dbc3d94a91852d797c731c2e97` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:40.595449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b252dbc3d94a91852d797c731c2e97)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:40.600578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:40.867238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b252dbc3d94a91852d797c731c2e97', 'request_id': 'req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:40.885610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b252dbc3d94a91852d797c731c2e97', 'request_id': 'req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.062644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b252dbc3d94a91852d797c731c2e97', 'request_id': 'req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.077664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] The token cache is being invalidate because project 47b252dbc3d94a91852d797c731c2e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:41.078027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.078738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.080393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1102/2197] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:40 2022] DELETE /identity/v3/projects/47b252dbc3d94a91852d797c731c2e97 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:41.087388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:41.099507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658107d-0547-4bb3-a817-47c654607c04 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:41.127122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:41.195247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:41.196192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:41.196770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:41.197737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] PATH_INFO: `/v3/users/31d60812b7f34271ae02396005c49652` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:41.199438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=31d60812b7f34271ae02396005c49652)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:41.202769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:41.534798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31d60812b7f34271ae02396005c49652'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.536185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31d60812b7f34271ae02396005c49652'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.536813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31d60812b7f34271ae02396005c49652'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.696104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31d60812b7f34271ae02396005c49652'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.708186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31d60812b7f34271ae02396005c49652'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:41.766327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1096/2198] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:41 2022] DELETE /identity/v3/users/31d60812b7f34271ae02396005c49652 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:42.753848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:42.753848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:42.766634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca4bd-82e8-4323-9e81-b0dce0346fbe admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:42.842864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:42.881775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:42.887287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:42.887287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:42.888536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:42.888791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] PATH_INFO: `/v3/projects/cb85eff649f74198830f70fc226c5ac7` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:42.890996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:42.891678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb85eff649f74198830f70fc226c5ac7)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:42.896544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:42.950503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042217e5-b265-439f-9608-f418774ee39d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:42.976740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:42.982650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:42.982650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:42.982650 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] PATH_INFO: `/v3/users/831139f58d58411cb73cb8076f2980da` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:42.983982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] RBAC: Authorizing `identity:delete_user(user_id=831139f58d58411cb73cb8076f2980da)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:42.998463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:43.176640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb85eff649f74198830f70fc226c5ac7', 'request_id': 'req-5a9cec31-0934-49d1-9882-4684ab00c03f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.209723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb85eff649f74198830f70fc226c5ac7', 'request_id': 'req-5a9cec31-0934-49d1-9882-4684ab00c03f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.398172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831139f58d58411cb73cb8076f2980da'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.403988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831139f58d58411cb73cb8076f2980da'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.405602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831139f58d58411cb73cb8076f2980da'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.437646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb85eff649f74198830f70fc226c5ac7', 'request_id': 'req-5a9cec31-0934-49d1-9882-4684ab00c03f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.514420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] The token cache is being invalidate because project cb85eff649f74198830f70fc226c5ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:43.521086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.530760 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5a9cec31-0934-49d1-9882-4684ab00c03f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.545834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1103/2199] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:42 2022] DELETE /identity/v3/projects/cb85eff649f74198830f70fc226c5ac7 => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:43.555378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:43.556066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:43.556830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:43.649196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831139f58d58411cb73cb8076f2980da'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.650673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:43.652925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:43.697026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-49bbda67-5ebd-4c58-8aae-ec2c37cdf03b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831139f58d58411cb73cb8076f2980da'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:43.760164 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1097/2200] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:38:42 2022] DELETE /identity/v3/users/831139f58d58411cb73cb8076f2980da => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:43.765115 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653bc1c-36e4-4f62-b5a0-657fe0cf2698 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:43.765429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653bc1c-36e4-4f62-b5a0-657fe0cf2698 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:43.765775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653bc1c-36e4-4f62-b5a0-657fe0cf2698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:43.817302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c653bc1c-36e4-4f62-b5a0-657fe0cf2698 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:43.819312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c653bc1c-36e4-4f62-b5a0-657fe0cf2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:43.854473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:43.913003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1104/2201] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:38:43.917258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:43.922228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:43.988348 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1098/2202] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:38:44.002136 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3d616-d277-4ded-b15c-233ab663de82 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:44.003141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3d616-d277-4ded-b15c-233ab663de82 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:44.003713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3d616-d277-4ded-b15c-233ab663de82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:44.034226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b3826-f356-46f1-b597-c7fa89626fcd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:44.059705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-93c3d616-d277-4ded-b15c-233ab663de82 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:44.067388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-93c3d616-d277-4ded-b15c-233ab663de82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:44.074246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1099/2203] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:44.075538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3d616-d277-4ded-b15c-233ab663de82 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:44.101436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:44.139925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:44.140813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:44.141210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:44.141587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:44.147230 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:44.202205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3d616-d277-4ded-b15c-233ab663de82 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:44.206793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:44.209392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:44.209793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:44.210201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:44.222730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:44.228530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:44.238278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1100/2204] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:44.240103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:44.253367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd6e18-5ed0-4d63-b612-07b7184b9595 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:44.270842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:44.272333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:44.273905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:44.274140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:44.274603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:44.275602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:44.276576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:38:44.276983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:44.278985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:44.303940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1101/2205] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:38:44.319136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1105/2206] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:44.319347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:44.339123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae9595-609b-4990-9b36-553a6d840b50 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:44.347317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:44.350229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:44.350642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:44.351239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:44.352175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:44.359468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:44.510229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1102/2207] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:38:44 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:44.515725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:44.548244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba4094-fd62-40af-a82c-36391ac4b203 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:44.719576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:44.720313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:44.721395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:44.721395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:44.721833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:44.726294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:44.968852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1106/2208] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:38:44 2022] POST /identity/v3/users => generated 305 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:44.974218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:44.992528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b888-b681-4780-9e51-9f9936edcb97 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:45.000350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:45.000690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:45.000858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:45.001233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:45.002154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:45.006488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:45.034430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1103/2209] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:45.039155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:45.060827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d0c6c-10ff-4e04-8d09-7b814e6bc319 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:45.068261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b87ca9-4248-4d40-99a1-ec56c73a9b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:45.069005 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b87ca9-4248-4d40-99a1-ec56c73a9b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:45.069352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b87ca9-4248-4d40-99a1-ec56c73a9b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:45.070516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b87ca9-4248-4d40-99a1-ec56c73a9b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:45.071727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b87ca9-4248-4d40-99a1-ec56c73a9b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:45.073661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b87ca9-4248-4d40-99a1-ec56c73a9b9e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:45.090320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1107/2210] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:45.096468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:45.117910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9d171-6f5f-47e0-8b95-1f9e03426699 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:45.127443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:45.128361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:45.128803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:45.130855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] PATH_INFO: `/v3/projects/4f22dd7d6bc545869be403bcc01ebd6d/users/4baaae6c28534ce8a85aa8bf915b3971/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:45.158716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f22dd7d6bc545869be403bcc01ebd6d, user_id=4baaae6c28534ce8a85aa8bf915b3971, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:45.166754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:45.321307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1104/2211] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:45 2022] PUT /identity/v3/projects/4f22dd7d6bc545869be403bcc01ebd6d/users/4baaae6c28534ce8a85aa8bf915b3971/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:45.331782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ade6a-6514-4672-b523-019bc0cda592 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:45.332158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ade6a-6514-4672-b523-019bc0cda592 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:45.332437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ade6a-6514-4672-b523-019bc0cda592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:45.388455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-be8ade6a-6514-4672-b523-019bc0cda592 None None] MFA Rules not processed for user `4baaae6c28534ce8a85aa8bf915b3971`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:45.391466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-be8ade6a-6514-4672-b523-019bc0cda592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:45.500092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:45.500753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:45.504514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:45.568437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1108/2212] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:45 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:45.571639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ade6a-6514-4672-b523-019bc0cda592 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:45.671908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:45.711562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ade6a-6514-4672-b523-019bc0cda592 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:45.726758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:45.727359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:45.727785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:45.728114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:45.728872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:45.730880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:45.744449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80985767-cabc-45cb-a6a7-e48d11fdc465 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:45.764486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:45.765154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:45.765480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:45.767871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] PATH_INFO: `/v3/projects/c28c72c2473844649fe8ac3c8245ac80` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:45.772543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28c72c2473844649fe8ac3c8245ac80)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:45.774870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:45.933191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1109/2213] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:38:45 2022] POST /identity/v3/projects => generated 407 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:38:45.939273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:46.025718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28c72c2473844649fe8ac3c8245ac80', 'request_id': 'req-e600306e-55dc-4f8f-804a-52467872506c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:46.036937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28c72c2473844649fe8ac3c8245ac80', 'request_id': 'req-e600306e-55dc-4f8f-804a-52467872506c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:46.048482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28c72c2473844649fe8ac3c8245ac80', 'request_id': 'req-e600306e-55dc-4f8f-804a-52467872506c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:46.074430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16739fc-99b5-4506-8929-02e1f770dd14 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:46.154088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] The token cache is being invalidate because project c28c72c2473844649fe8ac3c8245ac80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:38:46.154558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:46.156923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:38:46.166084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1105/2214] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:38:45 2022] DELETE /identity/v3/projects/c28c72c2473844649fe8ac3c8245ac80 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:46.200977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:46.201788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:46.202179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:46.202519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:46.203444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:46.206706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:46.387296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1110/2215] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:38:45 2022] POST /identity/v3/users => generated 310 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:38:46.391981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:46.393860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:46.423730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600306e-55dc-4f8f-804a-52467872506c admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:46.441340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:46.442421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:46.442701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:46.443011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:46.443649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:46.445368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:46.462133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1106/2216] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:38:46.464478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:46.480726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d656a-946d-4c21-b7ca-1e956ebed1de admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:46.495642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:46.496355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:46.497123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:46.497405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] PATH_INFO: `/v3/projects/82ecee1602d64b66bbeb7f0bd4116f30/users/dcf70bfac150458384f4ee013abf2da7/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:46.529835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ecee1602d64b66bbeb7f0bd4116f30, user_id=dcf70bfac150458384f4ee013abf2da7, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:46.533735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:46.660744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1111/2217] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:46 2022] PUT /identity/v3/projects/82ecee1602d64b66bbeb7f0bd4116f30/users/dcf70bfac150458384f4ee013abf2da7/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:46.662550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:46.687679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdb4ed-b70b-47a3-ae91-612efc64a732 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:46.704171 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:46.799703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf37d6-3a41-400c-8a26-e8d45c10198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:46.801430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf37d6-3a41-400c-8a26-e8d45c10198f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:46.801926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf37d6-3a41-400c-8a26-e8d45c10198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:46.802245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf37d6-3a41-400c-8a26-e8d45c10198f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:46.803123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf37d6-3a41-400c-8a26-e8d45c10198f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:46.805385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf37d6-3a41-400c-8a26-e8d45c10198f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:46.822049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1107/2218] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:38:46.825645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:46.839589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320734a-164e-4123-a983-ee5c97ec9b50 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:46.845434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:46.846657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:46.847143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:46.848222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] PATH_INFO: `/v3/projects/82ecee1602d64b66bbeb7f0bd4116f30/users/dcf70bfac150458384f4ee013abf2da7/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:46.852437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ecee1602d64b66bbeb7f0bd4116f30, user_id=dcf70bfac150458384f4ee013abf2da7, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:46.857973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:46.883469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1112/2219] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:38:46 2022] PUT /identity/v3/projects/82ecee1602d64b66bbeb7f0bd4116f30/users/dcf70bfac150458384f4ee013abf2da7/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:38:46.904320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:46.904703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:46.905098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:46.956105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] MFA Rules not processed for user `dcf70bfac150458384f4ee013abf2da7`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:46.962875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:47.056781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:38:47.172143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1108/2220] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:47.187713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:47.191792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:47.318948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79787f33-9917-4f5d-a216-065af7ce7127 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:47.324901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738827-914b-4613-a969-9f7aa6a8e8f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:47.326084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738827-914b-4613-a969-9f7aa6a8e8f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:47.326890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738827-914b-4613-a969-9f7aa6a8e8f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:47.327429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738827-914b-4613-a969-9f7aa6a8e8f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:47.341358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f738827-914b-4613-a969-9f7aa6a8e8f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:47.343811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f738827-914b-4613-a969-9f7aa6a8e8f0 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:47.371563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1113/2221] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:47 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:48.037036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:48.040644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:48.064378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8f0e-d1d0-43cf-92e2-7081c2fe6f1c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:48.071074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:48.071642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:48.072054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:48.072490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:48.076203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:48.222514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:48.222514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:48.255819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1109/2222] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:38:48 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:38:50.658165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a3ccb-59ed-4a84-a2d7-ea7587ed2da1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:50.658953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a3ccb-59ed-4a84-a2d7-ea7587ed2da1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:50.658953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a3ccb-59ed-4a84-a2d7-ea7587ed2da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:50.684976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ff1a3ccb-59ed-4a84-a2d7-ea7587ed2da1 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:38:50.687266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ff1a3ccb-59ed-4a84-a2d7-ea7587ed2da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:50.833369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1114/2223] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 24 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:38:50.855306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:38:50.857396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:38:50.988542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8751a2b-a667-4d66-a1c2-ff938fd06dc8 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:38:50.994252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cd11a-db9f-49ec-90ce-2404b8c2efa3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:38:50.994952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd11a-db9f-49ec-90ce-2404b8c2efa3 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:38:50.995235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd11a-db9f-49ec-90ce-2404b8c2efa3 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:38:50.995512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd11a-db9f-49ec-90ce-2404b8c2efa3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:38:51.006341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cd11a-db9f-49ec-90ce-2404b8c2efa3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:38:51.008602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cd11a-db9f-49ec-90ce-2404b8c2efa3 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:38:51.027837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1110/2224] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:38:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:39:00.195053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a3ccb-59ed-4a84-a2d7-ea7587ed2da1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:00.302042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a3ccb-59ed-4a84-a2d7-ea7587ed2da1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:00.308654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:00.310853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:00.311336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:00.312272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] PATH_INFO: `/v3/users/4baaae6c28534ce8a85aa8bf915b3971` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:00.314670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4baaae6c28534ce8a85aa8bf915b3971)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:00.318404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:00.355517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:00.592799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4baaae6c28534ce8a85aa8bf915b3971'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:00.593752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4baaae6c28534ce8a85aa8bf915b3971'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:00.604684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4baaae6c28534ce8a85aa8bf915b3971'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:00.605532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4baaae6c28534ce8a85aa8bf915b3971'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:00.684396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4baaae6c28534ce8a85aa8bf915b3971'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:00.729170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1115/2225] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:39:00 2022] DELETE /identity/v3/users/4baaae6c28534ce8a85aa8bf915b3971 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:00.730619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:00.731205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:00.731205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:00.755404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:00.757207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:00.867251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1111/2226] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:00.877213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:00.879858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:00.996192 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707af39d-b0d5-4946-8a97-4f11dee35ec7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:01.008383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:01.009458 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:01.010810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:01.011265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:01.042216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:01.046201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:01.071380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1116/2227] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:02.059161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:02.062758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:02.087014 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cde521-fb44-42e4-8d6a-06bedb741522 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:02.093433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:02.094224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:02.095175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:02.095459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] PATH_INFO: `/v3/projects/4f22dd7d6bc545869be403bcc01ebd6d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:02.097142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f22dd7d6bc545869be403bcc01ebd6d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:02.099760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:02.109216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:02.309033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f22dd7d6bc545869be403bcc01ebd6d', 'request_id': 'req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.320377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f22dd7d6bc545869be403bcc01ebd6d', 'request_id': 'req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.327795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f22dd7d6bc545869be403bcc01ebd6d', 'request_id': 'req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.436865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] The token cache is being invalidate because project 4f22dd7d6bc545869be403bcc01ebd6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:39:02.438124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.444670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.444670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1112/2228] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:39:02 2022] DELETE /identity/v3/projects/4f22dd7d6bc545869be403bcc01ebd6d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:02.447715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:02.464422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46540837-dd2d-48a3-9e0a-2e0d0c559769 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:02.560593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:02.561119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:02.561406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:02.561679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] PATH_INFO: `/v3/users/dcf70bfac150458384f4ee013abf2da7` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:02.562918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf70bfac150458384f4ee013abf2da7)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:02.564861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:02.827233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:02.834041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf70bfac150458384f4ee013abf2da7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.838040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf70bfac150458384f4ee013abf2da7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.847643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf70bfac150458384f4ee013abf2da7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.848896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf70bfac150458384f4ee013abf2da7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.964240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9795623f-0271-4a65-97ad-89109013418e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf70bfac150458384f4ee013abf2da7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:02.991404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1117/2229] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:39:02 2022] DELETE /identity/v3/users/dcf70bfac150458384f4ee013abf2da7 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:03.615967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:03.619906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:03.639091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0fc7e-bc05-421d-9d24-f40e9d1d77e5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:03.729026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:03.729797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:03.730325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:03.730790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] PATH_INFO: `/v3/projects/82ecee1602d64b66bbeb7f0bd4116f30` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:03.733191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ecee1602d64b66bbeb7f0bd4116f30)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:03.737383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:03.912611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ecee1602d64b66bbeb7f0bd4116f30', 'request_id': 'req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:03.924682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ecee1602d64b66bbeb7f0bd4116f30', 'request_id': 'req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:03.941609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ecee1602d64b66bbeb7f0bd4116f30', 'request_id': 'req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:04.083436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] The token cache is being invalidate because project 82ecee1602d64b66bbeb7f0bd4116f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:39:04.088438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:04.088438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8b72d6d6-c24d-4c23-b107-30f4dbdb2a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:04.088438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1113/2230] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:39:03 2022] DELETE /identity/v3/projects/82ecee1602d64b66bbeb7f0bd4116f30 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:04.098364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f159760-cddd-4918-abf1-1ec8bc64e15f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:04.100180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f159760-cddd-4918-abf1-1ec8bc64e15f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:04.100180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f159760-cddd-4918-abf1-1ec8bc64e15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:04.146396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4f159760-cddd-4918-abf1-1ec8bc64e15f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:04.149093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4f159760-cddd-4918-abf1-1ec8bc64e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:04.309957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1118/2231] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:04.314212 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637c8f-b204-42b8-959e-dab13697106d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:04.314717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637c8f-b204-42b8-959e-dab13697106d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:04.315220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637c8f-b204-42b8-959e-dab13697106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:04.351462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6b637c8f-b204-42b8-959e-dab13697106d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:04.353593 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b637c8f-b204-42b8-959e-dab13697106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:04.369289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1114/2232] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:04.375444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f159760-cddd-4918-abf1-1ec8bc64e15f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:04.393486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f159760-cddd-4918-abf1-1ec8bc64e15f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:04.404281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:04.405106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:04.405477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:04.405809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:04.406617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:39:04.407065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:04.409669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:04.440065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1119/2233] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:39:04.447423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b637c8f-b204-42b8-959e-dab13697106d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:04.460431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b637c8f-b204-42b8-959e-dab13697106d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:04.468949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:04.469970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:04.470403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:04.470785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:04.471792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:04.475275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:04.497257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:04.614155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1115/2234] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:39:04 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:39:04.620982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:04.634213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8416755-2174-45fe-b9cb-f9cbd78c510b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:04.749816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:04.750420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:04.750867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:04.751389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:04.751894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:04.754199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:04.898669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1120/2235] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:39:04 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:39:04.905937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:04.921043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa92a34-8b7a-43d9-aa12-e63a07720f42 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:04.926810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:04.927435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:04.927709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:04.927976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:04.928661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:04.930356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:04.942395 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1116/2236] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:39:04.947946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:04.966181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315174e3-680d-4e79-8be4-aad7a9d61cdc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:04.972477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac469f5-297d-41cf-9c28-1ba54e8ffd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:04.973515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac469f5-297d-41cf-9c28-1ba54e8ffd6a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:04.974164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac469f5-297d-41cf-9c28-1ba54e8ffd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:04.974616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac469f5-297d-41cf-9c28-1ba54e8ffd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:04.975548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac469f5-297d-41cf-9c28-1ba54e8ffd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:04.977612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac469f5-297d-41cf-9c28-1ba54e8ffd6a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:04.999049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1121/2237] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:39:05.004441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:05.016347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9f66f-da26-4e2a-9779-32da309efba9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:05.021982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:05.022820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:05.023079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:05.023373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] PATH_INFO: `/v3/projects/4d5800be31d04d7d91f5845a13062936/users/1e30fe70b525443cb4fda8d6ccf27c7f/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:05.042777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5800be31d04d7d91f5845a13062936, user_id=1e30fe70b525443cb4fda8d6ccf27c7f, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:05.045506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:05.207022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1117/2238] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:39:05 2022] PUT /identity/v3/projects/4d5800be31d04d7d91f5845a13062936/users/1e30fe70b525443cb4fda8d6ccf27c7f/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:05.234902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1afd6f-17cf-460f-ab4d-69370fc2f62f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:05.236296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1afd6f-17cf-460f-ab4d-69370fc2f62f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:05.236766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1afd6f-17cf-460f-ab4d-69370fc2f62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:05.283692 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5f1afd6f-17cf-460f-ab4d-69370fc2f62f None None] MFA Rules not processed for user `1e30fe70b525443cb4fda8d6ccf27c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:05.284654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5f1afd6f-17cf-460f-ab4d-69370fc2f62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:05.304403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:05.386686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1122/2239] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:05.403456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:05.405835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:05.477563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cec721-eb64-42af-be50-1a20a2357b96 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:05.481773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:05.482603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:05.483090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:05.483507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:05.494912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:05.496780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:05.510542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1118/2240] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:08.797143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fd716-643a-47d4-ae58-dc0a77ea08fa None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:08.797723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fd716-643a-47d4-ae58-dc0a77ea08fa None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:08.797723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fd716-643a-47d4-ae58-dc0a77ea08fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:08.826010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-875fd716-643a-47d4-ae58-dc0a77ea08fa None None] MFA Rules not processed for user `34b1955a937f4053967cf82cedd87a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:08.833472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-875fd716-643a-47d4-ae58-dc0a77ea08fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:08.905167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1123/2241] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:39:15.819507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:18.898046 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=46 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:22.898385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:22.900044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:22.994387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791a158-c289-4161-b880-7663f680fa3f service glance] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:23.001438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:23.002493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:23.002891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:23.004438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] PATH_INFO: `/v3/users/06c17524efb04a9e8ee52a2c388adc5f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:23.006756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c17524efb04a9e8ee52a2c388adc5f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:23.010588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:23.276056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c17524efb04a9e8ee52a2c388adc5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:23.276756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c17524efb04a9e8ee52a2c388adc5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:23.277371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c17524efb04a9e8ee52a2c388adc5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:23.468057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c17524efb04a9e8ee52a2c388adc5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:23.477849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c17524efb04a9e8ee52a2c388adc5f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:23.497980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1119/2242] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:39:22 2022] DELETE /identity/v3/users/06c17524efb04a9e8ee52a2c388adc5f => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:23.502765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:23.503920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:23.503920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:23.524669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:23.525897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:23.602240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1124/2243] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:39:23.612973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:23.614528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:23.695905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c06fe-119f-4337-9eb1-53bb51a99362 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:23.700825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11ea2f-12e6-44c5-a524-15ff67ed1a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:23.701855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ea2f-12e6-44c5-a524-15ff67ed1a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:23.703352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ea2f-12e6-44c5-a524-15ff67ed1a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:23.703802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11ea2f-12e6-44c5-a524-15ff67ed1a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:23.715526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11ea2f-12e6-44c5-a524-15ff67ed1a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:23.717039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11ea2f-12e6-44c5-a524-15ff67ed1a39 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:23.732035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1120/2244] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:39:24.258337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:24.261249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:24.275828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6fa2b-1724-4c75-b4aa-35ed0859d8d2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:24.281464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:24.282071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:24.282388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:24.282664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] PATH_INFO: `/v3/projects/2ed0a2d1734f414c8e34f24e8644a0e6` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:24.283875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed0a2d1734f414c8e34f24e8644a0e6)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:24.286823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:24.416864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed0a2d1734f414c8e34f24e8644a0e6', 'request_id': 'req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:24.422896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed0a2d1734f414c8e34f24e8644a0e6', 'request_id': 'req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:24.523538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed0a2d1734f414c8e34f24e8644a0e6', 'request_id': 'req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:24.538981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] The token cache is being invalidate because project 2ed0a2d1734f414c8e34f24e8644a0e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:39:24.539240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:24.539873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ae450ab-d654-45c4-a4c2-dc7bb96701a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:24.541440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1125/2245] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:39:24 2022] DELETE /identity/v3/projects/2ed0a2d1734f414c8e34f24e8644a0e6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:39:24.546633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dedf73-0228-42f9-84f4-d67098ff2c8f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:24.547287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dedf73-0228-42f9-84f4-d67098ff2c8f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:24.547287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dedf73-0228-42f9-84f4-d67098ff2c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:24.572798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-04dedf73-0228-42f9-84f4-d67098ff2c8f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:24.574112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-04dedf73-0228-42f9-84f4-d67098ff2c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:24.595188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:24.656161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1121/2246] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:24.666452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511c4df-bb9b-4067-9471-f870b6a4a108 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:24.666788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511c4df-bb9b-4067-9471-f870b6a4a108 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:24.667009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511c4df-bb9b-4067-9471-f870b6a4a108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:24.691672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8511c4df-bb9b-4067-9471-f870b6a4a108 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:24.693072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8511c4df-bb9b-4067-9471-f870b6a4a108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:24.702163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1126/2247] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:24.707487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dedf73-0228-42f9-84f4-d67098ff2c8f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:24.718597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dedf73-0228-42f9-84f4-d67098ff2c8f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:24.724616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:24.725239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:24.725513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:24.725844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:24.726483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:39:24.726782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:24.728640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:24.742974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1122/2248] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:39:24.749170 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511c4df-bb9b-4067-9471-f870b6a4a108 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:24.760574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511c4df-bb9b-4067-9471-f870b6a4a108 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:24.768416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:24.769265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:24.769695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:24.769969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:24.770703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:24.772689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:24.877939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1127/2249] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:39:24 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:39:24.883862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:24.898407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe531a3-dabd-4fa0-96c4-dafb11362cc2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:25.016439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:25.017104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:25.017375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:25.017646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:25.018359 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:25.021953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:25.173660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1123/2250] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:39:24 2022] POST /identity/v3/users => generated 287 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:39:25.178684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:25.191313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cb821-3322-469b-9db6-84033a0cf6c0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:25.196993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:25.199888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:25.199888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:25.199888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:25.199888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:25.200309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:25.219204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1128/2251] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:39:25.233171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:25.244524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc56a5-48c0-41e1-bc3a-1893a5d539be admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:25.250767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b658b3-1d6d-4269-8ed5-e2fe11f5cf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:25.251659 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b658b3-1d6d-4269-8ed5-e2fe11f5cf12 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:25.252017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b658b3-1d6d-4269-8ed5-e2fe11f5cf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:25.252392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b658b3-1d6d-4269-8ed5-e2fe11f5cf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:25.254821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b658b3-1d6d-4269-8ed5-e2fe11f5cf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:25.257374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b658b3-1d6d-4269-8ed5-e2fe11f5cf12 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:25.273537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1124/2252] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:39:25.278761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:25.296948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45d908-c1ef-4266-81f7-db48760b7224 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:25.305128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:25.305949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:25.306368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:25.316793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] PATH_INFO: `/v3/projects/445b4b1878bb4fddb22ed9c9937ca62f/users/b7c947382d1147f4ac92f845064877b8/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:25.334902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445b4b1878bb4fddb22ed9c9937ca62f, user_id=b7c947382d1147f4ac92f845064877b8, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:25.337813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:25.496708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1129/2253] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:39:25 2022] PUT /identity/v3/projects/445b4b1878bb4fddb22ed9c9937ca62f/users/b7c947382d1147f4ac92f845064877b8/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:25.508418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9388c39-926e-4364-a005-1ebbc033f4b5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:25.508793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9388c39-926e-4364-a005-1ebbc033f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:25.509070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9388c39-926e-4364-a005-1ebbc033f4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:25.554374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d9388c39-926e-4364-a005-1ebbc033f4b5 None None] MFA Rules not processed for user `b7c947382d1147f4ac92f845064877b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:25.556556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d9388c39-926e-4364-a005-1ebbc033f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:25.647222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1125/2254] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:25.660917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:25.662832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:25.698457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:25.742963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df815-13a1-44ef-9aa3-0b857ab427e6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:25.746936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a5bb-5691-418d-9e0e-3fad09d45afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:25.747569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a5bb-5691-418d-9e0e-3fad09d45afd service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:25.747846 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a5bb-5691-418d-9e0e-3fad09d45afd service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:25.748124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a5bb-5691-418d-9e0e-3fad09d45afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:25.758895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a5bb-5691-418d-9e0e-3fad09d45afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:25.760364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a5bb-5691-418d-9e0e-3fad09d45afd service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:25.771511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1130/2255] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:25.807264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9388c39-926e-4364-a005-1ebbc033f4b5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:25.837375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:25.896494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9388c39-926e-4364-a005-1ebbc033f4b5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:25.902352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:25.903175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:25.903632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:25.904090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] PATH_INFO: `/v3/users/b7c947382d1147f4ac92f845064877b8` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:25.906153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c947382d1147f4ac92f845064877b8)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:25.909526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:26.186177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c947382d1147f4ac92f845064877b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:26.187039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c947382d1147f4ac92f845064877b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:26.187940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c947382d1147f4ac92f845064877b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:26.295751 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c947382d1147f4ac92f845064877b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:26.310143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c947382d1147f4ac92f845064877b8'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:26.337508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1126/2256] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:39:25 2022] DELETE /identity/v3/users/b7c947382d1147f4ac92f845064877b8 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:39:26.342447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:26.343093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:26.343552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:26.375117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:26.392507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:26.512893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1131/2257] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:26.524474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:26.526260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:26.622477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a410efb-ed5b-4b5c-a3c7-37c2b88ec04f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:26.627078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:26.627884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:26.628790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:26.629113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:26.658676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:26.660849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:26.673889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1127/2258] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:27.381075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:27.383119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:27.400838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c31817-d2e4-4c89-b5f4-cb51e35094dd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:27.408994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:27.409128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:27.409318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:27.410373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] PATH_INFO: `/v3/projects/445b4b1878bb4fddb22ed9c9937ca62f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:27.412027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=445b4b1878bb4fddb22ed9c9937ca62f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:27.414832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:27.656693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445b4b1878bb4fddb22ed9c9937ca62f', 'request_id': 'req-6d1b765e-af6f-40fa-9d57-5230ff26db06'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:27.674596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445b4b1878bb4fddb22ed9c9937ca62f', 'request_id': 'req-6d1b765e-af6f-40fa-9d57-5230ff26db06'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:27.833369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445b4b1878bb4fddb22ed9c9937ca62f', 'request_id': 'req-6d1b765e-af6f-40fa-9d57-5230ff26db06'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:27.862383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] The token cache is being invalidate because project 445b4b1878bb4fddb22ed9c9937ca62f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:39:27.863984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:27.865828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:27.872740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1132/2259] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:39:27 2022] DELETE /identity/v3/projects/445b4b1878bb4fddb22ed9c9937ca62f => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:36.993108 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:37.855619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:40.701723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:40.711601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173936a-1842-4c79-a04a-193aa62b74aa service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:40.775413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:40.776042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:40.776318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:40.776590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:40.778505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:40.863879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:40.865409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:40.880194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1128/2260] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:44.558616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:44.560128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:44.573896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b765e-af6f-40fa-9d57-5230ff26db06 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:44.578281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1daa8a-f19d-469f-8cd2-f0f93400b561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:44.578929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1daa8a-f19d-469f-8cd2-f0f93400b561 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:44.579266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1daa8a-f19d-469f-8cd2-f0f93400b561 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:44.579552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1daa8a-f19d-469f-8cd2-f0f93400b561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:44.582264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5d1daa8a-f19d-469f-8cd2-f0f93400b561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:44.597861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1daa8a-f19d-469f-8cd2-f0f93400b561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:44.599374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1daa8a-f19d-469f-8cd2-f0f93400b561 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:44.615391 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1133/2261] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:46.573488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:46.575121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:46.661341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3752de-7971-4091-a61b-44734825f1dd service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:46.666844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:46.667513 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:46.667793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:46.668056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] PATH_INFO: `/v3/users/1e30fe70b525443cb4fda8d6ccf27c7f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:46.669505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e30fe70b525443cb4fda8d6ccf27c7f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:46.671826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:46.968667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:46.973369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e30fe70b525443cb4fda8d6ccf27c7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:46.974222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e30fe70b525443cb4fda8d6ccf27c7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:46.975010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e30fe70b525443cb4fda8d6ccf27c7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:47.089404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e30fe70b525443cb4fda8d6ccf27c7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:47.098927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e30fe70b525443cb4fda8d6ccf27c7f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:47.119711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1129/2262] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:39:46 2022] DELETE /identity/v3/users/1e30fe70b525443cb4fda8d6ccf27c7f => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:47.124661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048560da-232d-4898-b07c-7aa435425a2b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:47.125195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048560da-232d-4898-b07c-7aa435425a2b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:47.125195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048560da-232d-4898-b07c-7aa435425a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:47.150429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-048560da-232d-4898-b07c-7aa435425a2b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:47.155470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-048560da-232d-4898-b07c-7aa435425a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:47.229973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1134/2263] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:47.241105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:47.242609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:47.307102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372129-f7dd-4797-bc23-ed43926ed178 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:47.310912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078988-66d4-40de-922c-5ea09372ce08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:47.311542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078988-66d4-40de-922c-5ea09372ce08 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:47.311819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078988-66d4-40de-922c-5ea09372ce08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:47.312098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078988-66d4-40de-922c-5ea09372ce08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:47.324237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44078988-66d4-40de-922c-5ea09372ce08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:47.326642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44078988-66d4-40de-922c-5ea09372ce08 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:47.345848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1130/2264] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:47.928204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048560da-232d-4898-b07c-7aa435425a2b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:47.929839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-048560da-232d-4898-b07c-7aa435425a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:47.947066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048560da-232d-4898-b07c-7aa435425a2b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:47.950299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:47.950960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:47.951352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:47.951558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] PATH_INFO: `/v3/projects/4d5800be31d04d7d91f5845a13062936` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:47.952869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5800be31d04d7d91f5845a13062936)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:47.955683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:48.102145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5800be31d04d7d91f5845a13062936', 'request_id': 'req-d9f7398b-583a-4eee-a900-9736efdfe8d2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:48.111936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5800be31d04d7d91f5845a13062936', 'request_id': 'req-d9f7398b-583a-4eee-a900-9736efdfe8d2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:48.220847 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5800be31d04d7d91f5845a13062936', 'request_id': 'req-d9f7398b-583a-4eee-a900-9736efdfe8d2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:48.235819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] The token cache is being invalidate because project 4d5800be31d04d7d91f5845a13062936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:39:48.236316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:48.237105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:39:48.239681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1135/2265] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:39:47 2022] DELETE /identity/v3/projects/4d5800be31d04d7d91f5845a13062936 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:39:53.367617 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27053-8915-4627-b4ea-9effeb76dcc4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:53.368176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27053-8915-4627-b4ea-9effeb76dcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:53.368282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27053-8915-4627-b4ea-9effeb76dcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:53.405623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-95f27053-8915-4627-b4ea-9effeb76dcc4 None None] MFA Rules not processed for user `3570472f8c2b42b480ec1d1220414bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:53.407981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-95f27053-8915-4627-b4ea-9effeb76dcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:53.511542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1131/2266] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:39:57.821948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:39:57.824850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:57.942941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7398b-583a-4eee-a900-9736efdfe8d2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:39:57.952698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:39:57.952698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:57.952698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:57.952698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:57.953285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:57.962829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:39:57.965654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:39:57.967885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:39:57.981771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1136/2267] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:39:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:39:59.043396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:39:59.044193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:39:59.044533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:39:59.079568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] MFA Rules not processed for user `c888090dd9f246a79c6be54faf480ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:39:59.082660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:39:59.180344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1132/2268] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:39:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:40:01.854245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:01.867933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad792b2e-9ee1-4bbd-8a49-89394e31f170 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:01.874266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:01.874951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:01.875331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:01.875921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:01.878361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:01.952852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:01.954463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:01.966160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1137/2269] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:40:04.405033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:04.406548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:04.442736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:40:04.515600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:04.517131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:04.585377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eec23e-aaca-4368-8314-d252dbd80149 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:04.589879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f1c66-ef4a-4a13-a2da-60755def5fa6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:04.590626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1c66-ef4a-4a13-a2da-60755def5fa6 service glance] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:04.590910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1c66-ef4a-4a13-a2da-60755def5fa6 service glance] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:04.591185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1c66-ef4a-4a13-a2da-60755def5fa6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:04.593077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-733f1c66-ef4a-4a13-a2da-60755def5fa6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:04.610811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f1c66-ef4a-4a13-a2da-60755def5fa6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:04.612814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f1c66-ef4a-4a13-a2da-60755def5fa6 service glance] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:04.628971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1133/2270] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:40:04.656173 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796b629-507e-45d0-9552-bd1f723fce5e service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:04.661605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:04.662372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:04.662881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:04.663588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:04.676209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:04.679100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:04.692993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1138/2271] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:40:11.262819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:11.263746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:11.263746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:11.287778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] MFA Rules not processed for user `7b946ecd87434ad6a00890887d234281`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:40:11.289368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:11.297073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1134/2272] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:40:11.497981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:11.521874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05de55-062e-4e8d-be84-39cfa526fd31 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:11.531921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:11.532555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:11.532836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:11.533223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:11.536306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:11.640909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:11.644389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:11.666817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1139/2273] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:40:11.775376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:11.779238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:11.801171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55992fd7-af09-4da3-a74f-e530fc25251a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:11.806141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:11.806802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:11.807081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:11.808627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:11.810687 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:11.824382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:11.826866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:11.853563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1135/2274] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:40:12.331256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:12.333430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:12.407808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:40:12.470166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bacb97-38b1-4a7e-a27f-989d84bef02e service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:12.491466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:12.492587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:12.493222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:12.493861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:12.498671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:12.516773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:12.519338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:12.540313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1140/2275] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:40:17.207624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:17.228946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedf959-de4b-4d90-b0f8-ec9de9ec89c8 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:17.243034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:17.244459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:17.246520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:17.247338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:17.250965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:17.271192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:17.274118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:17.293666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1136/2276] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:40:22.531023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=16 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:40:27.284978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=35 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:40:30.729207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:30.756851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01192-7252-4c18-95b8-6e813b83ab35 service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:30.770305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b944ba-b0fa-4ebb-9b2c-63227528db00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:30.771675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b944ba-b0fa-4ebb-9b2c-63227528db00 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:30.771827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b944ba-b0fa-4ebb-9b2c-63227528db00 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:30.772584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b944ba-b0fa-4ebb-9b2c-63227528db00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:30.775999 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-18b944ba-b0fa-4ebb-9b2c-63227528db00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:30.807550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b944ba-b0fa-4ebb-9b2c-63227528db00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:30.822577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b944ba-b0fa-4ebb-9b2c-63227528db00 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:30.854653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1141/2277] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:40:31.869896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:31.894626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abeacf-b494-472b-a425-633189168c69 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:31.910781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c93d45-cbd8-4aa5-b1c1-750e91281135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:31.912506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c93d45-cbd8-4aa5-b1c1-750e91281135 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:31.913007 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c93d45-cbd8-4aa5-b1c1-750e91281135 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:31.913287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c93d45-cbd8-4aa5-b1c1-750e91281135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:31.916271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-61c93d45-cbd8-4aa5-b1c1-750e91281135 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:31.933457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c93d45-cbd8-4aa5-b1c1-750e91281135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:31.935338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c93d45-cbd8-4aa5-b1c1-750e91281135 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:31.949530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1137/2278] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:40:37.398102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:37.398991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:37.398991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:37.442638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] MFA Rules not processed for user `34b1955a937f4053967cf82cedd87a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:40:37.443890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:37.452244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1142/2279] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:40:41.944090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:40:47.432665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=9 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:40:53.111748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d2559-211f-42d4-a522-62f745288d98 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:53.127138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d2559-211f-42d4-a522-62f745288d98 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:53.127138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d2559-211f-42d4-a522-62f745288d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:53.139669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f24d2559-211f-42d4-a522-62f745288d98 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:40:53.140821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f24d2559-211f-42d4-a522-62f745288d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:53.226689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1138/2280] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 24 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:40:53.585363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:53.586461 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:53.688558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f47ff-c61b-43b3-a3bb-b3512d053e4a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:53.695389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:53.696091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:53.696348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:53.696795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:53.708281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:53.710352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:53.722041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1143/2281] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:40:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:40:53.884145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d2559-211f-42d4-a522-62f745288d98 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:40:53.899447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d2559-211f-42d4-a522-62f745288d98 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:40:53.906753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:40:53.907472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:40:53.908194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:40:53.908194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:40:53.911537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:40:53.926737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:40:53.928369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:40:53.941596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1139/2282] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:40:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:01.736212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:01.821960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16680803-ca33-45e9-991f-df521b5c1772 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:01.828260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:01.830583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:01.831056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:01.831512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:01.833880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:02.090343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:02.091876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:02.119381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:02.125037 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1144/2283] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:03.936534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=31 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:07.562791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:07.659817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd97f-d25b-4672-be82-50c23d13940a service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:07.668950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:07.670629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:07.671118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:07.671605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:07.676167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:07.697978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:07.700148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:07.713543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1140/2284] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:07.800573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:07.803331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:07.822839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698cbc0-bc21-4fe7-8a9c-f0e7d5af2cbc service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:07.831661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:07.832823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:07.833316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:07.833816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] PATH_INFO: `/v3/users/de720e2d8a0f4b5f898e00ce90befd58` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:07.836231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] RBAC: Authorizing `identity:delete_user(user_id=de720e2d8a0f4b5f898e00ce90befd58)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:07.840070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:08.138409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de720e2d8a0f4b5f898e00ce90befd58'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:08.139581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de720e2d8a0f4b5f898e00ce90befd58'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:08.161400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de720e2d8a0f4b5f898e00ce90befd58'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:08.163216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de720e2d8a0f4b5f898e00ce90befd58'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:08.276869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de720e2d8a0f4b5f898e00ce90befd58'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:08.318237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1145/2285] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:41:07 2022] DELETE /identity/v3/users/de720e2d8a0f4b5f898e00ce90befd58 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:08.786619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:08.814878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46963b15-9720-4259-8b37-91b2659c8b8d service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:08.914993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:08.920498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:08.933058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:08.933831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:08.934225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:08.934557 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] PATH_INFO: `/v3/projects/853f7f0f1616478c9744f93293be553e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:08.935898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=853f7f0f1616478c9744f93293be553e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:08.937979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:09.044660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb19195-0eae-4017-b9d5-34d84f12a3be admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:09.050614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:09.051285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:09.051558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:09.051827 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] PATH_INFO: `/v3/users/3570472f8c2b42b480ec1d1220414bec` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:09.053043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3570472f8c2b42b480ec1d1220414bec)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:09.055021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:09.128214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '853f7f0f1616478c9744f93293be553e', 'request_id': 'req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.134658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '853f7f0f1616478c9744f93293be553e', 'request_id': 'req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.142461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '853f7f0f1616478c9744f93293be553e', 'request_id': 'req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.277085 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] The token cache is being invalidate because project 853f7f0f1616478c9744f93293be553e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:41:09.280333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.280333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.281085 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1141/2286] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:41:08 2022] DELETE /identity/v3/projects/853f7f0f1616478c9744f93293be553e => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:09.287119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:09.288709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:09.396317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3570472f8c2b42b480ec1d1220414bec'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.397169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3570472f8c2b42b480ec1d1220414bec'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.397720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:09.407145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3570472f8c2b42b480ec1d1220414bec'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.408012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3570472f8c2b42b480ec1d1220414bec'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.416228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf55ba-1a0d-4db6-ae7d-388f87bd7410 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:09.429501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:09.429501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:09.429941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:09.430375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] PATH_INFO: `/v3/users/97061137f48845398bb10504c4bf3577` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:09.432825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97061137f48845398bb10504c4bf3577)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:09.441863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:09.500052 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3570472f8c2b42b480ec1d1220414bec'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.518615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:09.532812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1146/2287] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:41:08 2022] DELETE /identity/v3/users/3570472f8c2b42b480ec1d1220414bec => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:09.810170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97061137f48845398bb10504c4bf3577'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.810983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97061137f48845398bb10504c4bf3577'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.811702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97061137f48845398bb10504c4bf3577'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.856615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:09.858332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:09.950860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97061137f48845398bb10504c4bf3577'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.952231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6150-b8c3-49ae-92e4-f765ae15bad8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:09.960744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:09.961414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:09.961711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:09.961922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9fccbeac-1b43-4295-b535-2e8df9e4d144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97061137f48845398bb10504c4bf3577'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:09.962121 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] PATH_INFO: `/v3/projects/7af0efe201074713a9076e220de8ea61` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:09.963288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af0efe201074713a9076e220de8ea61)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:09.965265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:09.987977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1142/2288] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:41:09 2022] DELETE /identity/v3/users/97061137f48845398bb10504c4bf3577 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:10.117956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af0efe201074713a9076e220de8ea61', 'request_id': 'req-653d089e-4bd9-4274-9142-c663f01c2e94'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.130935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af0efe201074713a9076e220de8ea61', 'request_id': 'req-653d089e-4bd9-4274-9142-c663f01c2e94'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.226179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af0efe201074713a9076e220de8ea61', 'request_id': 'req-653d089e-4bd9-4274-9142-c663f01c2e94'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.252962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] The token cache is being invalidate because project 7af0efe201074713a9076e220de8ea61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:41:10.253485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.254723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-653d089e-4bd9-4274-9142-c663f01c2e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.257814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1147/2289] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:41:09 2022] DELETE /identity/v3/projects/7af0efe201074713a9076e220de8ea61 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:10.265107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa83c5-3cd9-42f2-89ac-10d67835014d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:10.265443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa83c5-3cd9-42f2-89ac-10d67835014d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:10.265699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa83c5-3cd9-42f2-89ac-10d67835014d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:10.351302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c9aa83c5-3cd9-42f2-89ac-10d67835014d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:10.351302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c9aa83c5-3cd9-42f2-89ac-10d67835014d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:10.422776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1143/2290] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:41:10.432108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3705b-f92a-47bc-a92f-632c9fd4a8b7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:10.432427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3705b-f92a-47bc-a92f-632c9fd4a8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:10.432668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3705b-f92a-47bc-a92f-632c9fd4a8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:10.456025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-25b3705b-f92a-47bc-a92f-632c9fd4a8b7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:10.457387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-25b3705b-f92a-47bc-a92f-632c9fd4a8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:10.466228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1148/2291] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:41:10.471441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa83c5-3cd9-42f2-89ac-10d67835014d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:10.482918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa83c5-3cd9-42f2-89ac-10d67835014d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:10.489029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:10.489788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:10.490294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:10.490609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:10.491428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:41:10.491801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:10.493634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:10.509369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1144/2292] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:41:10.515953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3705b-f92a-47bc-a92f-632c9fd4a8b7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:10.526838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3705b-f92a-47bc-a92f-632c9fd4a8b7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:10.532153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:10.533539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:10.533825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:10.534141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:10.534856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:10.536819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:10.577882 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:10.580846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:10.600366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d2a7d-6b5a-42ab-8b2d-faa5b4d5dd15 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:10.614913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:10.614913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:10.614913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:10.614913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] PATH_INFO: `/v3/projects/78caa9a337004170aa7cf8919813151b` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:10.615511 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78caa9a337004170aa7cf8919813151b)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:10.615511 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:10.642535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:10.750227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1149/2293] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:41:10 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:41:10.755878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:10.795811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78caa9a337004170aa7cf8919813151b', 'request_id': 'req-67a2cf8e-d749-4825-aff0-ffe31262d732'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.805462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78caa9a337004170aa7cf8919813151b', 'request_id': 'req-67a2cf8e-d749-4825-aff0-ffe31262d732'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.820837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78caa9a337004170aa7cf8919813151b', 'request_id': 'req-67a2cf8e-d749-4825-aff0-ffe31262d732'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.843421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:10.862746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9584e-5ec4-406d-9b13-634984c661ea admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:10.901682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] The token cache is being invalidate because project 78caa9a337004170aa7cf8919813151b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:41:10.902135 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.903197 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:10.905980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1145/2294] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:41:10 2022] DELETE /identity/v3/projects/78caa9a337004170aa7cf8919813151b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:10.950225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:10.951223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:10.951666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:10.952088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:10.953151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:10.957099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:11.064359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1150/2295] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:41:10 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:41:11.067482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:11.070858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:11.093238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2cf8e-d749-4825-aff0-ffe31262d732 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:11.101199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:11.101916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:11.102323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:11.102718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:11.103465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:11.105220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:11.126874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1146/2296] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:11.129191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:11.143790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75626b1e-8aee-48d2-a5dc-71fdc7f1af7e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:11.153689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4aedbb-9726-4da0-bb2b-7f5b67e5a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:11.159341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aedbb-9726-4da0-bb2b-7f5b67e5a5fd admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:11.159341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aedbb-9726-4da0-bb2b-7f5b67e5a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:11.159341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4aedbb-9726-4da0-bb2b-7f5b67e5a5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:11.159341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4aedbb-9726-4da0-bb2b-7f5b67e5a5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:11.161531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4aedbb-9726-4da0-bb2b-7f5b67e5a5fd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:11.183175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1151/2297] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:11.188163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:11.201567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312e872-80a8-4543-b6ad-b434f8909e8f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:11.208645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:11.209558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:11.210055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:11.210490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] PATH_INFO: `/v3/projects/a0fe699cd3ef4690a65566213e0b7ce6/users/bccbd0fd2c2d40cf8786af088dc65b0a/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:11.229393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0fe699cd3ef4690a65566213e0b7ce6, user_id=bccbd0fd2c2d40cf8786af088dc65b0a, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:11.232737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:11.323413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1147/2298] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:41:11 2022] PUT /identity/v3/projects/a0fe699cd3ef4690a65566213e0b7ce6/users/bccbd0fd2c2d40cf8786af088dc65b0a/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:11.336558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633c865-eb91-4fdc-b0e2-3884e99cf2d5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:11.336824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633c865-eb91-4fdc-b0e2-3884e99cf2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:11.337108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633c865-eb91-4fdc-b0e2-3884e99cf2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:11.389373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c633c865-eb91-4fdc-b0e2-3884e99cf2d5 None None] MFA Rules not processed for user `bccbd0fd2c2d40cf8786af088dc65b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:11.390680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c633c865-eb91-4fdc-b0e2-3884e99cf2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:11.513207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1152/2299] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:11.526964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:11.529104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:11.667475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2c5e8-34f8-48d5-b925-753b1eebc2aa admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:11.671857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958401b6-f0ef-4df4-8410-3cb41d8cce31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:11.672627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958401b6-f0ef-4df4-8410-3cb41d8cce31 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:11.673699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958401b6-f0ef-4df4-8410-3cb41d8cce31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:11.674615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958401b6-f0ef-4df4-8410-3cb41d8cce31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:11.685167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958401b6-f0ef-4df4-8410-3cb41d8cce31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:11.689792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958401b6-f0ef-4df4-8410-3cb41d8cce31 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:11.699001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1148/2300] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:21.509180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:21.693763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:23.277112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633c865-eb91-4fdc-b0e2-3884e99cf2d5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:23.398479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:23.399043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:23.399043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:23.401380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633c865-eb91-4fdc-b0e2-3884e99cf2d5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:23.408608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:23.409850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:23.410439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:23.410945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] PATH_INFO: `/v3/users/bccbd0fd2c2d40cf8786af088dc65b0a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:23.413298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccbd0fd2c2d40cf8786af088dc65b0a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:23.417398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:23.432172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:23.433430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:23.502613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1149/2301] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:41:23.516559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:23.518050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:23.604001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45579f96-3074-4ec9-902b-651f45507dbb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:23.609743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a88b2-c60f-490e-b5e4-a40ce9617462 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:23.610710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a88b2-c60f-490e-b5e4-a40ce9617462 service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:23.612357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a88b2-c60f-490e-b5e4-a40ce9617462 service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:23.612357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a88b2-c60f-490e-b5e4-a40ce9617462 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:23.624154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a88b2-c60f-490e-b5e4-a40ce9617462 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:23.627368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a88b2-c60f-490e-b5e4-a40ce9617462 service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:23.639676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1150/2302] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:41:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:41:23.701018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccbd0fd2c2d40cf8786af088dc65b0a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:23.702165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccbd0fd2c2d40cf8786af088dc65b0a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:23.714862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccbd0fd2c2d40cf8786af088dc65b0a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:23.715733 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccbd0fd2c2d40cf8786af088dc65b0a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:23.884167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccbd0fd2c2d40cf8786af088dc65b0a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:23.945035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1153/2303] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:41:23 2022] DELETE /identity/v3/users/bccbd0fd2c2d40cf8786af088dc65b0a => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:23.951976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:23.952471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:23.952960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:23.994383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:24.000207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:24.165270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1151/2304] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:24.182862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:24.185210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:24.341438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec3a5d-fd18-43bc-8bdf-ebd21f9f95b3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:24.347109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69075-0cb6-4d06-bb2f-59c05ae3a291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:24.347746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69075-0cb6-4d06-bb2f-59c05ae3a291 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:24.348115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69075-0cb6-4d06-bb2f-59c05ae3a291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:24.348538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69075-0cb6-4d06-bb2f-59c05ae3a291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:24.362616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69075-0cb6-4d06-bb2f-59c05ae3a291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:24.364313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69075-0cb6-4d06-bb2f-59c05ae3a291 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:24.383492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1154/2305] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:25.072503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:25.075619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:25.128614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60899ca0-0e3a-4329-8369-8f16f36cad29 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:25.145475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:25.147570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:25.147972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:25.148269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] PATH_INFO: `/v3/projects/a0fe699cd3ef4690a65566213e0b7ce6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:25.150678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0fe699cd3ef4690a65566213e0b7ce6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:25.152990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:25.198081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:25.362367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0fe699cd3ef4690a65566213e0b7ce6', 'request_id': 'req-88b8f855-c2b5-4428-8051-ebe13a85c443'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:25.373955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0fe699cd3ef4690a65566213e0b7ce6', 'request_id': 'req-88b8f855-c2b5-4428-8051-ebe13a85c443'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:25.392462 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0fe699cd3ef4690a65566213e0b7ce6', 'request_id': 'req-88b8f855-c2b5-4428-8051-ebe13a85c443'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:25.502704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] The token cache is being invalidate because project a0fe699cd3ef4690a65566213e0b7ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:41:25.503165 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:25.504324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-88b8f855-c2b5-4428-8051-ebe13a85c443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:25.506021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1152/2306] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:41:25 2022] DELETE /identity/v3/projects/a0fe699cd3ef4690a65566213e0b7ce6 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:25.516797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff427996-6378-49d1-b9f8-8921b78c7486 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:25.518282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff427996-6378-49d1-b9f8-8921b78c7486 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:25.518451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff427996-6378-49d1-b9f8-8921b78c7486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:25.543434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ff427996-6378-49d1-b9f8-8921b78c7486 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:25.544713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ff427996-6378-49d1-b9f8-8921b78c7486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:25.551113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:25.678162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1155/2307] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:25.690942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909b9b5-a24f-4a0e-8687-1d1f71d40d29 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:25.691598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909b9b5-a24f-4a0e-8687-1d1f71d40d29 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:25.692148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909b9b5-a24f-4a0e-8687-1d1f71d40d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:25.740269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0909b9b5-a24f-4a0e-8687-1d1f71d40d29 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:25.742843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0909b9b5-a24f-4a0e-8687-1d1f71d40d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:25.767103 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1153/2308] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:25.769414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff427996-6378-49d1-b9f8-8921b78c7486 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:25.789665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff427996-6378-49d1-b9f8-8921b78c7486 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:25.798803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:25.799832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:25.800313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:25.800803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:25.802271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:41:25.802854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:25.806560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:25.834251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1156/2309] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:41:25.841450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909b9b5-a24f-4a0e-8687-1d1f71d40d29 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:25.858979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909b9b5-a24f-4a0e-8687-1d1f71d40d29 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:25.868142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:25.869336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:25.869879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:25.870407 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:25.871576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:25.875326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:26.034504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1154/2310] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:41:25 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:41:26.038328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:26.058076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ffb19-d368-4ef6-b141-7301beaa87ec admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:26.192035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:26.194042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:26.194922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:26.195316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:26.196503 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:26.200274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:26.404739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:26.407506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1157/2311] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:41:26 2022] POST /identity/v3/users => generated 282 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:41:26.430143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3fffe-a3d3-43b7-b94e-98ff8747e1e1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:26.444750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:26.449495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:26.449495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:26.449495 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:26.450132 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:26.452855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:26.479719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1155/2312] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:26.485372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:26.530375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718f94d-9036-459a-9d41-764c22575441 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:26.530375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3e202-df5a-433b-b65c-c9bff8df5bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:26.531446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3e202-df5a-433b-b65c-c9bff8df5bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:26.531632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3e202-df5a-433b-b65c-c9bff8df5bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:26.532795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3e202-df5a-433b-b65c-c9bff8df5bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:26.534496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3e202-df5a-433b-b65c-c9bff8df5bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:26.542167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3e202-df5a-433b-b65c-c9bff8df5bd4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:26.567317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1158/2313] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:26.570598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:26.599605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f60a3-1743-4d01-91d2-dcfbfe8f8fa4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:26.609467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:26.610707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:26.611775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:26.612286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] PATH_INFO: `/v3/projects/79f7e336078d4ee98a5dfe71d02fef33/users/91005f22ec98494382c8d22e6d2381aa/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:26.648607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f7e336078d4ee98a5dfe71d02fef33, user_id=91005f22ec98494382c8d22e6d2381aa, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:26.653105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:26.757084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1156/2314] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:41:26 2022] PUT /identity/v3/projects/79f7e336078d4ee98a5dfe71d02fef33/users/91005f22ec98494382c8d22e6d2381aa/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:26.776099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddb31b-f2f8-43cd-9b57-2efbdebe0b85 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:26.776608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddb31b-f2f8-43cd-9b57-2efbdebe0b85 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:26.777134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddb31b-f2f8-43cd-9b57-2efbdebe0b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:26.863794 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0fddb31b-f2f8-43cd-9b57-2efbdebe0b85 None None] MFA Rules not processed for user `91005f22ec98494382c8d22e6d2381aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:26.865614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0fddb31b-f2f8-43cd-9b57-2efbdebe0b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:26.977681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1159/2315] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:26.984060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:27.077776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08c178-69a9-4484-bf6f-841909d8caf2 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:27.085891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:27.086736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:27.087083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:27.087428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:27.088525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:27.091331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:27.619803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1157/2316] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:41:26 2022] POST /identity/v3/projects => generated 365 bytes in 636 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:41:27.634485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddb31b-f2f8-43cd-9b57-2efbdebe0b85 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:27.655296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddb31b-f2f8-43cd-9b57-2efbdebe0b85 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:27.661763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:27.765543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:27.767233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:27.767716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:27.768199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:27.769379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:27.773274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:27.948751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1160/2317] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:41:27 2022] POST /identity/v3/users => generated 289 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:41:27.956404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:27.969843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642a403-52b4-4c02-89d1-6fe59f9d5fc7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:27.975627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:27.976262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:27.976548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:27.976830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:27.977661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:27.979420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:27.998123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1158/2318] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:28.005709 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:28.023901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baf1f1-ce10-4113-b4b9-3076f8c1be06 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:28.039774 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:28.041399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:28.042607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:28.042607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] PATH_INFO: `/v3/projects/0d01b4e82f7f4f4f9f351566b84b5560/users/daa42a765209490abe1da0b1162ae849/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:28.072085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d01b4e82f7f4f4f9f351566b84b5560, user_id=daa42a765209490abe1da0b1162ae849, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:28.077329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:28.186062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1161/2319] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:41:28 2022] PUT /identity/v3/projects/0d01b4e82f7f4f4f9f351566b84b5560/users/daa42a765209490abe1da0b1162ae849/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:28.188760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:28.206982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808af61-4590-4b62-92b9-74dbce8dc03a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:28.237795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:28.305961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfcefc-c2f1-41d0-b9b9-a61e4c6b0829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:28.307431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfcefc-c2f1-41d0-b9b9-a61e4c6b0829 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:28.308133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfcefc-c2f1-41d0-b9b9-a61e4c6b0829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:28.308523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfcefc-c2f1-41d0-b9b9-a61e4c6b0829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:28.309530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfcefc-c2f1-41d0-b9b9-a61e4c6b0829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:28.311697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfcefc-c2f1-41d0-b9b9-a61e4c6b0829 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:28.339882 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:28.341430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1159/2320] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:41:28.358323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217736b-4a5c-49df-bbf1-dc53947d2759 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:28.368406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:28.369077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:28.369441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:28.369824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] PATH_INFO: `/v3/projects/0d01b4e82f7f4f4f9f351566b84b5560/users/daa42a765209490abe1da0b1162ae849/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:28.377681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d01b4e82f7f4f4f9f351566b84b5560, user_id=daa42a765209490abe1da0b1162ae849, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:28.380536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:28.406358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1162/2321] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:41:28 2022] PUT /identity/v3/projects/0d01b4e82f7f4f4f9f351566b84b5560/users/daa42a765209490abe1da0b1162ae849/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:28.426644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377310c0-8409-4003-aa9a-97f70b23c1fb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:28.427096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377310c0-8409-4003-aa9a-97f70b23c1fb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:28.427561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377310c0-8409-4003-aa9a-97f70b23c1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:28.483521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-377310c0-8409-4003-aa9a-97f70b23c1fb None None] MFA Rules not processed for user `daa42a765209490abe1da0b1162ae849`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:28.486186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-377310c0-8409-4003-aa9a-97f70b23c1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:28.621291 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1160/2322] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:41:28.645449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:28.648229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:28.799912 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080411ce-7826-45c1-8fa6-2a8456e472dd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:28.809762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:28.810930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:28.811519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:28.812067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:28.835427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:28.837599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:28.870614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1163/2323] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:41:29.176726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377310c0-8409-4003-aa9a-97f70b23c1fb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:29.200139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377310c0-8409-4003-aa9a-97f70b23c1fb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:29.225636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b40982-8331-4b21-82a2-831e9413da55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:29.225636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b40982-8331-4b21-82a2-831e9413da55 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:29.226201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b40982-8331-4b21-82a2-831e9413da55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:29.226201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b40982-8331-4b21-82a2-831e9413da55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:29.302306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b40982-8331-4b21-82a2-831e9413da55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:29.303894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b40982-8331-4b21-82a2-831e9413da55 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:29.318512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1161/2324] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:41:38.847214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:39.312248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=82 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:45.205312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:45.371367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1b6a-da75-401a-8f89-21fe925bdc1e service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:45.389055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:45.389775 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:45.390183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:45.390625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] PATH_INFO: `/v3/users/91005f22ec98494382c8d22e6d2381aa` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:45.393189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91005f22ec98494382c8d22e6d2381aa)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:45.400452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:45.859722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91005f22ec98494382c8d22e6d2381aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:45.861182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91005f22ec98494382c8d22e6d2381aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:45.869335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91005f22ec98494382c8d22e6d2381aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:45.870242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91005f22ec98494382c8d22e6d2381aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:45.989520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91005f22ec98494382c8d22e6d2381aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:46.012338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1164/2325] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:41:45 2022] DELETE /identity/v3/users/91005f22ec98494382c8d22e6d2381aa => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:46.017558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:46.018216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:46.018216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:46.038579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:46.039840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:46.127093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1162/2326] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:46.137645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:46.139309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:46.209922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950e94e-18e2-4244-9278-af57b779e8f4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:46.214927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:46.216522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:46.216522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:46.216522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:46.232339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:46.233854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:46.246209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1165/2327] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:46.918613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:46.921334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:46.949089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405588-6cbe-4acf-844c-c690cdafcdec None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:46.959753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:46.960449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:46.960760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:46.961037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] PATH_INFO: `/v3/projects/79f7e336078d4ee98a5dfe71d02fef33` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:46.962423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f7e336078d4ee98a5dfe71d02fef33)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:46.964914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:47.189005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f7e336078d4ee98a5dfe71d02fef33', 'request_id': 'req-1a2309f2-f60b-439b-abce-dd3d49e18171'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.201025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f7e336078d4ee98a5dfe71d02fef33', 'request_id': 'req-1a2309f2-f60b-439b-abce-dd3d49e18171'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.213833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f7e336078d4ee98a5dfe71d02fef33', 'request_id': 'req-1a2309f2-f60b-439b-abce-dd3d49e18171'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.340236 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] The token cache is being invalidate because project 79f7e336078d4ee98a5dfe71d02fef33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:41:47.340550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.341096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.342755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1163/2328] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:41:46 2022] DELETE /identity/v3/projects/79f7e336078d4ee98a5dfe71d02fef33 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:47.349861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:47.368792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fc2a-c293-4c1a-91f6-5a82d5b8636e service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:47.398025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:47.461785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:47.462545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:47.462933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:47.463300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] PATH_INFO: `/v3/users/daa42a765209490abe1da0b1162ae849` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:47.464866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa42a765209490abe1da0b1162ae849)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:47.467091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:47.730721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa42a765209490abe1da0b1162ae849'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.731398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa42a765209490abe1da0b1162ae849'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.739810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa42a765209490abe1da0b1162ae849'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.740543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa42a765209490abe1da0b1162ae849'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.885578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa42a765209490abe1da0b1162ae849'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:47.919845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1166/2329] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:41:47 2022] DELETE /identity/v3/users/daa42a765209490abe1da0b1162ae849 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:48.167304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:48.169663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:48.262543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2309f2-f60b-439b-abce-dd3d49e18171 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:48.311116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:48.323736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd375d-85b2-4180-b706-d10cec831652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:48.324358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd375d-85b2-4180-b706-d10cec831652 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:48.324634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd375d-85b2-4180-b706-d10cec831652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:48.324910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd375d-85b2-4180-b706-d10cec831652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:48.326795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-aafd375d-85b2-4180-b706-d10cec831652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:48.337840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd375d-85b2-4180-b706-d10cec831652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:48.339356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd375d-85b2-4180-b706-d10cec831652 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:48.349155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1164/2330] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:48.559973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:48.562512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:48.646633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa49f8-84cd-4199-b743-8a4d7fdb823d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:48.651440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:48.652089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:48.652370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:48.652639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] PATH_INFO: `/v3/projects/0d01b4e82f7f4f4f9f351566b84b5560` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:48.653788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d01b4e82f7f4f4f9f351566b84b5560)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:48.655751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:48.856309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d01b4e82f7f4f4f9f351566b84b5560', 'request_id': 'req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:48.866936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d01b4e82f7f4f4f9f351566b84b5560', 'request_id': 'req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:48.918507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e20557-27eb-4051-96d6-ab86bf82b687 None None] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:48.918748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e20557-27eb-4051-96d6-ab86bf82b687 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:48.918974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e20557-27eb-4051-96d6-ab86bf82b687 None None] PATH_INFO: `/` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:48.920360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1165/2331] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:41:48.927359 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:48.927664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:48.927930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:48.944546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d01b4e82f7f4f4f9f351566b84b5560', 'request_id': 'req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:48.952735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:48.955467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:48.968012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] The token cache is being invalidate because project 0d01b4e82f7f4f4f9f351566b84b5560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:41:48.968665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:48.970425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-085eeccf-1d45-4ac3-9cf5-61f9d6e8471b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:48.973016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1167/2332] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:41:48 2022] DELETE /identity/v3/projects/0d01b4e82f7f4f4f9f351566b84b5560 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:48.978751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b22b4-41e1-4d87-af2e-7080a2d79186 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:48.979609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b22b4-41e1-4d87-af2e-7080a2d79186 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:48.980265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b22b4-41e1-4d87-af2e-7080a2d79186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.013743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-111b22b4-41e1-4d87-af2e-7080a2d79186 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:49.016390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-111b22b4-41e1-4d87-af2e-7080a2d79186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:49.072866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1166/2333] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:41:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:49.086960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.088533 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:49.091864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1168/2334] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:49.101981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae81e2-75aa-48af-8403-49b8d85ab3d3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.102379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae81e2-75aa-48af-8403-49b8d85ab3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.102824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae81e2-75aa-48af-8403-49b8d85ab3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.133251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-62ae81e2-75aa-48af-8403-49b8d85ab3d3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:49.134835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-62ae81e2-75aa-48af-8403-49b8d85ab3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:49.145798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1169/2335] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:49.151018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae81e2-75aa-48af-8403-49b8d85ab3d3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.162531 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae81e2-75aa-48af-8403-49b8d85ab3d3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.168926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.176149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15785212-0dca-4849-9fe4-5b54f65c5b91 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.176350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.176350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.176350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.176350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:41:49.176350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.176350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.177073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.177073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.177073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.177073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.187676 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1170/2336] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:41:49.192826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.193031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.195523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.204580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465f72-2e2f-404e-a676-db2a00bfb89d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.209906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70984e0-17d5-4427-af14-e8d4d9327deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.210642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70984e0-17d5-4427-af14-e8d4d9327deb admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.210996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70984e0-17d5-4427-af14-e8d4d9327deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.211342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70984e0-17d5-4427-af14-e8d4d9327deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.212053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70984e0-17d5-4427-af14-e8d4d9327deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.213019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1167/2337] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:41:49.215255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70984e0-17d5-4427-af14-e8d4d9327deb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.237010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:49.468505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1171/2338] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:41:49 2022] POST /identity/v3/projects => generated 381 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:41:49.474385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.486160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133c872-32c3-4f08-9272-e2e7682a8979 service glance] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.553826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.554611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.554966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.555241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.555892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.558521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.574101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87ba5-656c-4a76-aecb-7121827edbfc None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.574986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87ba5-656c-4a76-aecb-7121827edbfc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.575206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87ba5-656c-4a76-aecb-7121827edbfc None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.576604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1172/2339] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:41:49.582413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.582701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.582980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.607260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] MFA Rules not processed for user `48364ccf19434c4c9c09b4e5ce4bfc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:49.608498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:49.681566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1173/2340] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:41:49.695801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.697545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:49.711524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3c614-ced2-4ed9-b72b-2fc0f54165d4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.717833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.720745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1168/2341] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:41:49 2022] POST /identity/v3/users => generated 291 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:41:49.720916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.720916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.720916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.729833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.730941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.733597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.747226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe0d9b-54e0-4a62-9a20-5df61b72d9fc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.747536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1174/2342] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:41:49.755860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.757435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.757927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.758455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.759627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.762734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.776652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1169/2343] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:49.782460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.803188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c256881-1993-4df9-9a75-44281a34075e service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.817327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d12ed7-4f9d-44cc-bbfb-284dc35e595a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.817327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12ed7-4f9d-44cc-bbfb-284dc35e595a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.817327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12ed7-4f9d-44cc-bbfb-284dc35e595a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.817327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12ed7-4f9d-44cc-bbfb-284dc35e595a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.818194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d12ed7-4f9d-44cc-bbfb-284dc35e595a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.821235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d12ed7-4f9d-44cc-bbfb-284dc35e595a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.845287 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1175/2344] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:49.849050 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:49.860767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda6eea-ac29-47c1-929d-56ce9c6a19bd admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:49.866712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00ce5f-542a-44b8-9433-821f22b3aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:49.867420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00ce5f-542a-44b8-9433-821f22b3aed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.867804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00ce5f-542a-44b8-9433-821f22b3aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.868080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00ce5f-542a-44b8-9433-821f22b3aed6 admin admin] PATH_INFO: `/v3/projects/e577cc46fa9c4e4981dcfd55db7267dc/users/0a3ebf703c234e7ba3b25076872f6ebf/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.883485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8a4d-429c-480e-ae32-c69c0c18e1c9 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.883707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8a4d-429c-480e-ae32-c69c0c18e1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.883905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb8a4d-429c-480e-ae32-c69c0c18e1c9 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.885849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1176/2345] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 24 14:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 14:41:49.886922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00ce5f-542a-44b8-9433-821f22b3aed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e577cc46fa9c4e4981dcfd55db7267dc, user_id=0a3ebf703c234e7ba3b25076872f6ebf, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:49.889815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00ce5f-542a-44b8-9433-821f22b3aed6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:49.897886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b518c6-4d83-448e-ab47-df2a0b7a1580 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.898373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b518c6-4d83-448e-ab47-df2a0b7a1580 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.898658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b518c6-4d83-448e-ab47-df2a0b7a1580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:49.920199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e6b518c6-4d83-448e-ab47-df2a0b7a1580 None None] MFA Rules not processed for user `613e985e94c54913a5e6b93367157fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:49.921444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e6b518c6-4d83-448e-ab47-df2a0b7a1580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:49.979340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1170/2346] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:41:49 2022] PUT /identity/v3/projects/e577cc46fa9c4e4981dcfd55db7267dc/users/0a3ebf703c234e7ba3b25076872f6ebf/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:41:49.991962 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:49.992251 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:49.992600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:50.019878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1177/2347] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 24 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:50.031429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b518c6-4d83-448e-ab47-df2a0b7a1580 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:50.035148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] MFA Rules not processed for user `0a3ebf703c234e7ba3b25076872f6ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:50.037270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:50.051927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b518c6-4d83-448e-ab47-df2a0b7a1580 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:50.115392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:50.129367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1171/2348] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:41:50.151455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:50.152959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:50.153845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8f68b-e4c2-4898-9c42-9c4e00d75814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:50.154615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8f68b-e4c2-4898-9c42-9c4e00d75814 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:50.154990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8f68b-e4c2-4898-9c42-9c4e00d75814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:50.155386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8f68b-e4c2-4898-9c42-9c4e00d75814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:50.239590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34166b77-2c6c-46dc-aa01-da6b8b7d5e88 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:50.245796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100596c-9804-4269-9937-cf083340513d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:50.247048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100596c-9804-4269-9937-cf083340513d service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:50.247492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100596c-9804-4269-9937-cf083340513d service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:50.247941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100596c-9804-4269-9937-cf083340513d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:50.262832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100596c-9804-4269-9937-cf083340513d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:50.265053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100596c-9804-4269-9937-cf083340513d service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:50.277771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1172/2349] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:50.301977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8f68b-e4c2-4898-9c42-9c4e00d75814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:50.310440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8f68b-e4c2-4898-9c42-9c4e00d75814 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:50.313906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100596c-9804-4269-9937-cf083340513d service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:50.332491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1178/2350] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:50.420506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100596c-9804-4269-9937-cf083340513d service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:50.426211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:50.426885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:50.427161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:50.427454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] PATH_INFO: `/v3/users/0a3ebf703c234e7ba3b25076872f6ebf` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:50.428808 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3ebf703c234e7ba3b25076872f6ebf)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:50.430709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:50.808689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3ebf703c234e7ba3b25076872f6ebf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:50.810561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3ebf703c234e7ba3b25076872f6ebf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:50.811859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3ebf703c234e7ba3b25076872f6ebf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:50.917342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3ebf703c234e7ba3b25076872f6ebf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:50.923631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:50.933287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3ebf703c234e7ba3b25076872f6ebf'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:50.970287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1173/2351] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:41:50 2022] DELETE /identity/v3/users/0a3ebf703c234e7ba3b25076872f6ebf => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:50.977244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:50.978665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:50.980634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:51.023354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:51.025638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:51.158203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1179/2352] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:51.180304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:51.180304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:51.286752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba0453-0c41-4957-8b7c-bef17f9b40d6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:51.291756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:51.292587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:51.293010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:51.293454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:51.307999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:51.314084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:51.326087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1174/2353] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:51.347491 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:52.028294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:52.028294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:52.052851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7c16e-cec3-40f9-b9d7-f76364fdf478 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:52.059918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:52.060816 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:52.061285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:52.061960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] PATH_INFO: `/v3/projects/e577cc46fa9c4e4981dcfd55db7267dc` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:52.064684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e577cc46fa9c4e4981dcfd55db7267dc)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:52.067515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:52.259093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e577cc46fa9c4e4981dcfd55db7267dc', 'request_id': 'req-0c542a3a-f05d-4f94-a0ea-56b5479950a5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:52.267820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e577cc46fa9c4e4981dcfd55db7267dc', 'request_id': 'req-0c542a3a-f05d-4f94-a0ea-56b5479950a5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:52.386443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:52.395899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de443df-f9cb-4b2b-bed0-71ecd7479973 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:52.455070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0b546-fa38-4913-8be6-1a38d9bc7f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:52.455718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0b546-fa38-4913-8be6-1a38d9bc7f95 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:52.455991 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0b546-fa38-4913-8be6-1a38d9bc7f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:52.456264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0b546-fa38-4913-8be6-1a38d9bc7f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:52.458451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8de0b546-fa38-4913-8be6-1a38d9bc7f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:52.567310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0b546-fa38-4913-8be6-1a38d9bc7f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:52.568837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0b546-fa38-4913-8be6-1a38d9bc7f95 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:52.582182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1175/2354] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:41:52.586310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e577cc46fa9c4e4981dcfd55db7267dc', 'request_id': 'req-0c542a3a-f05d-4f94-a0ea-56b5479950a5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:52.613074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] The token cache is being invalidate because project e577cc46fa9c4e4981dcfd55db7267dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:41:52.614074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:52.616373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c542a3a-f05d-4f94-a0ea-56b5479950a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:41:52.620948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1180/2355] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:41:52 2022] DELETE /identity/v3/projects/e577cc46fa9c4e4981dcfd55db7267dc => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:52.632426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06038a1c-fc40-40bb-8887-826f9064698e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:52.633121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06038a1c-fc40-40bb-8887-826f9064698e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:52.633648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06038a1c-fc40-40bb-8887-826f9064698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:52.664884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-06038a1c-fc40-40bb-8887-826f9064698e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:52.667917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-06038a1c-fc40-40bb-8887-826f9064698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:52.693319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1176/2356] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:52.707581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3ad6-e49f-410e-9bba-b7a33bb543ca None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:52.707868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3ad6-e49f-410e-9bba-b7a33bb543ca None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:52.708137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3ad6-e49f-410e-9bba-b7a33bb543ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:52.737369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-472c3ad6-e49f-410e-9bba-b7a33bb543ca None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:52.738994 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-472c3ad6-e49f-410e-9bba-b7a33bb543ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:52.759789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1181/2357] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:41:52.765262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06038a1c-fc40-40bb-8887-826f9064698e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:52.794082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06038a1c-fc40-40bb-8887-826f9064698e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:52.805555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:52.806895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:52.809083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:52.810996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:52.819577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:52.822360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:41:52.822942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:52.829974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:52.872324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1177/2358] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:41:52.879741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c3ad6-e49f-410e-9bba-b7a33bb543ca None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:52.891756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:52.901399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c3ad6-e49f-410e-9bba-b7a33bb543ca None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:52.912365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:52.914766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:52.915516 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:52.916208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:52.917485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:52.923363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:53.108932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1182/2359] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:41:52 2022] POST /identity/v3/projects => generated 375 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:41:53.115233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:53.132431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989cce5-b2c7-497f-8176-dd009a7e5783 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:53.231798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:53.232715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:53.233131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:53.233543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:53.235261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:53.239022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:53.448286 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1178/2360] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:41:53 2022] POST /identity/v3/users => generated 288 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:41:53.454367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:53.467815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b156bfd-6f05-454b-8117-60e5a749adef admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:53.475403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:53.475883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:53.476255 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:53.476675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:53.477495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:53.479799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:53.494786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1183/2361] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:41:53.499707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:53.520157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f237-b187-492b-bdfa-c6d37306bac5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:53.528183 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac00273-977e-458a-9d73-4cd2bbc9ec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:53.529199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac00273-977e-458a-9d73-4cd2bbc9ec56 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:53.529582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac00273-977e-458a-9d73-4cd2bbc9ec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:53.530033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac00273-977e-458a-9d73-4cd2bbc9ec56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:53.531049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac00273-977e-458a-9d73-4cd2bbc9ec56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:53.534573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac00273-977e-458a-9d73-4cd2bbc9ec56 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:53.561681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1179/2362] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:41:53.563862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:53.579543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a202ef5-76ec-4c22-8fdf-ed1bdd5d8461 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:53.587336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:53.588314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:53.588599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:53.588924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] PATH_INFO: `/v3/projects/30419ac6bc344c5c86d13e0a2e545b7f/users/3d200c61facf4da785aca18751ac2bc0/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:53.615270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=30419ac6bc344c5c86d13e0a2e545b7f, user_id=3d200c61facf4da785aca18751ac2bc0, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:53.620463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:53.715838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1184/2363] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:41:53 2022] PUT /identity/v3/projects/30419ac6bc344c5c86d13e0a2e545b7f/users/3d200c61facf4da785aca18751ac2bc0/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:41:53.728167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e143ac6-139f-4c81-9fc6-07a336eb288c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:53.728660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e143ac6-139f-4c81-9fc6-07a336eb288c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:53.729233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e143ac6-139f-4c81-9fc6-07a336eb288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:53.783416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9e143ac6-139f-4c81-9fc6-07a336eb288c None None] MFA Rules not processed for user `3d200c61facf4da785aca18751ac2bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:53.785562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9e143ac6-139f-4c81-9fc6-07a336eb288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:53.891871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1180/2364] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:41:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:53.908950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:53.911150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:54.007538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51666240-cddd-41ce-bf97-b9ac55689bdc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:54.015520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:54.017731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:54.018323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:54.018818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:54.030304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:54.032220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:54.043659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1185/2365] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:41:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:41:55.779989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:55.780548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:55.780548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:55.803504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:41:55.804801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:55.832061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:55.904736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1181/2366] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:41:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:41:55.938576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:41:55.940123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:41:55.961401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:41:56.036821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf21bd9-b53b-4bf3-8001-c2a9143e2eb5 service glance] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:41:56.045591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:41:56.046400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:41:56.052745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:41:56.052745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:41:56.059485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:41:56.061573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:41:56.072384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1186/2367] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:41:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:42:01.120126 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:01.121640 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:01.335925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf08b8-e363-4b72-b1b9-505de80f2ed0 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:01.353448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:01.356150 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:01.378019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:01.378019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:01.378019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:01.378019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:01.385308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:01.445205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:01.450520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:01.498038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1182/2368] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:01.513321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0d7e6-2e65-4e43-b7b0-0b295555939e service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:01.561632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:01.565219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:01.567493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:01.574916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] PATH_INFO: `/v3/users/7b946ecd87434ad6a00890887d234281` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:01.574916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b946ecd87434ad6a00890887d234281)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:01.574916 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:01.863908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b946ecd87434ad6a00890887d234281'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:01.865145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b946ecd87434ad6a00890887d234281'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:01.879981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b946ecd87434ad6a00890887d234281'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:01.881409 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b946ecd87434ad6a00890887d234281'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:02.012749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4e5a62fe-3ceb-4273-9b24-b6384a1185cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b946ecd87434ad6a00890887d234281'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:02.051443 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1187/2369] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:01 2022] DELETE /identity/v3/users/7b946ecd87434ad6a00890887d234281 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:02.507705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:02.522616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc91af-f588-4ea7-8a3d-677d2305345a service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:02.660365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:02.661111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:02.661454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:02.662021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] PATH_INFO: `/v3/projects/d2bc03440f454c15abb4aab9bf6e2374` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:02.663832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2bc03440f454c15abb4aab9bf6e2374)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:02.666672 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:02.834503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2bc03440f454c15abb4aab9bf6e2374', 'request_id': 'req-2aafedf4-34d6-4389-9215-4e9bd6cc2246'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:02.850811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2bc03440f454c15abb4aab9bf6e2374', 'request_id': 'req-2aafedf4-34d6-4389-9215-4e9bd6cc2246'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:02.855547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:02.867604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2bc03440f454c15abb4aab9bf6e2374', 'request_id': 'req-2aafedf4-34d6-4389-9215-4e9bd6cc2246'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:02.990598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] The token cache is being invalidate because project d2bc03440f454c15abb4aab9bf6e2374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:02.991158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:02.992611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-2aafedf4-34d6-4389-9215-4e9bd6cc2246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:03.000373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1183/2370] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:02 2022] DELETE /identity/v3/projects/d2bc03440f454c15abb4aab9bf6e2374 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:03.002358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d170-2819-480a-a398-30d8c18c4aa0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:03.003028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d170-2819-480a-a398-30d8c18c4aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:03.003028 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d170-2819-480a-a398-30d8c18c4aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:03.036025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-75c5d170-2819-480a-a398-30d8c18c4aa0 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:03.038133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-75c5d170-2819-480a-a398-30d8c18c4aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:03.068191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:03.218093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1188/2371] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:03.224211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd09e9d-16ad-4824-962b-3b936bf8ac52 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:03.224211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd09e9d-16ad-4824-962b-3b936bf8ac52 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:03.224500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd09e9d-16ad-4824-962b-3b936bf8ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:03.265266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fbd09e9d-16ad-4824-962b-3b936bf8ac52 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:03.266873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fbd09e9d-16ad-4824-962b-3b936bf8ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:03.283322 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1184/2372] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:03.284997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d170-2819-480a-a398-30d8c18c4aa0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:03.297828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d170-2819-480a-a398-30d8c18c4aa0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:03.314759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:03.317285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:03.318999 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:03.319787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:03.324431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:42:03.327079 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:03.334747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:03.378412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1189/2373] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:42:03.379510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd09e9d-16ad-4824-962b-3b936bf8ac52 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:03.394024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd09e9d-16ad-4824-962b-3b936bf8ac52 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:03.400094 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:03.400760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:03.401106 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:03.401400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:03.401933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:03.404138 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:03.561339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1185/2374] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:03 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:03.569825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:03.636267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af10fc1-d58b-4568-8ba1-cf96cd0c1392 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:03.846770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:03.847568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:03.847993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:03.848447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:03.849568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:03.852129 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:04.000840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1190/2375] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:03 2022] POST /identity/v3/users => generated 292 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:04.005800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:04.021446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc7bb2-c499-4166-9fa6-b5b58fcf61c4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:04.029043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:04.030289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:04.030586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:04.030845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:04.032715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:04.034923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:04.053064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1186/2376] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:04.057859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:04.092980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1016a-8ce3-4760-8fe6-5c4a89e5b76c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:04.116054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3330ea-010e-4d59-8183-bae2e0421ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:04.116691 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3330ea-010e-4d59-8183-bae2e0421ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:04.116971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3330ea-010e-4d59-8183-bae2e0421ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:04.117347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3330ea-010e-4d59-8183-bae2e0421ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:04.117964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3330ea-010e-4d59-8183-bae2e0421ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:04.119789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3330ea-010e-4d59-8183-bae2e0421ffe admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:04.147007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1191/2377] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:04.148335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:04.168150 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378e9e-e341-49f0-b574-8ac50196bc6d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:04.176131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:04.180243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:04.180243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:04.180243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] PATH_INFO: `/v3/projects/205a3fd429dd4f56918861c582938551/users/e85c39cd135a43f5a9c58b57c25b9872/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:04.210125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205a3fd429dd4f56918861c582938551, user_id=e85c39cd135a43f5a9c58b57c25b9872, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:04.217118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:04.353896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1187/2378] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:04 2022] PUT /identity/v3/projects/205a3fd429dd4f56918861c582938551/users/e85c39cd135a43f5a9c58b57c25b9872/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:04.355658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:04.355950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:04.356225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:04.417499 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] MFA Rules not processed for user `e85c39cd135a43f5a9c58b57c25b9872`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:04.420400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:04.587030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1192/2379] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:04.604567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:04.606762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:04.785720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d13bc9-ccfa-481a-91c5-6889204e8e65 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:04.815372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:04.816186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:04.816664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:04.817012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:04.837522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:04.837522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:04.848948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1188/2380] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:14.114626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:14.116411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:14.150696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:14.203112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0519d8-9633-49ef-a382-9d9bf3ab09d9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:14.209650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:14.210767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:14.211187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:14.211632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:14.214657 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:14.306573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:14.325177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:14.325177 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1193/2381] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:42:14.842946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=64,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:15.646718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:15.649192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:15.686515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc9fd2-3639-4610-bcb1-584fa672a8e7 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:15.692761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:15.707376 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4397e71-9d0d-4f1f-a649-d5a6b973867c service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:15.710551 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:15.711497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:15.711994 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:15.712302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] PATH_INFO: `/v3/users/3d200c61facf4da785aca18751ac2bc0` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:15.714614 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d200c61facf4da785aca18751ac2bc0)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:15.717316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:15.724448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:15.725123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:15.725417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:15.725681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] PATH_INFO: `/v3/users/e85c39cd135a43f5a9c58b57c25b9872` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:15.726951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e85c39cd135a43f5a9c58b57c25b9872)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:15.728971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:15.998276 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e85c39cd135a43f5a9c58b57c25b9872'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:15.999341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e85c39cd135a43f5a9c58b57c25b9872'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.012853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e85c39cd135a43f5a9c58b57c25b9872'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.015515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e85c39cd135a43f5a9c58b57c25b9872'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.065887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d200c61facf4da785aca18751ac2bc0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.068484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d200c61facf4da785aca18751ac2bc0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.070304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d200c61facf4da785aca18751ac2bc0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.144675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e8115292-d581-4e58-b816-13d1815aaf8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e85c39cd135a43f5a9c58b57c25b9872'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.181939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1194/2382] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:15 2022] DELETE /identity/v3/users/e85c39cd135a43f5a9c58b57c25b9872 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:16.188921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:16.189404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:16.189877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:16.225515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:16.227662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:16.277739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:16.298415 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d200c61facf4da785aca18751ac2bc0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.307766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-28068081-c887-406d-9237-7b08bf139201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d200c61facf4da785aca18751ac2bc0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:16.317479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1195/2383] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:16.331870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:16.333607 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:16.349198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1189/2384] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:15 2022] DELETE /identity/v3/users/3d200c61facf4da785aca18751ac2bc0 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:16.357463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6a02-aceb-422e-9c2b-7b7b5509771b None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:16.357903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6a02-aceb-422e-9c2b-7b7b5509771b None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:16.358439 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6a02-aceb-422e-9c2b-7b7b5509771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:16.400538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a7dd6a02-aceb-422e-9c2b-7b7b5509771b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:16.402710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a7dd6a02-aceb-422e-9c2b-7b7b5509771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:16.415440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908bc9a-0d03-4597-8490-39df124097b0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:16.477849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:16.478894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:16.480227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:16.480778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:16.511088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1190/2385] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:16.523701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd6a02-aceb-422e-9c2b-7b7b5509771b None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:16.539798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd6a02-aceb-422e-9c2b-7b7b5509771b None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:16.555771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51283f-8827-4305-abb2-30c69043825a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:16.565886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51283f-8827-4305-abb2-30c69043825a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:16.565886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51283f-8827-4305-abb2-30c69043825a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:16.565886 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51283f-8827-4305-abb2-30c69043825a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:16.572357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51283f-8827-4305-abb2-30c69043825a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:16.575488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51283f-8827-4305-abb2-30c69043825a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:16.596523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1191/2386] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:42:16.600590 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:16.603741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:16.630574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1196/2387] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:42:17.158725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:17.159819 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:17.160081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:17.198064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:17.199618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:17.367361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1192/2388] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 24 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:17.403295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:17.406791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:17.513002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:17.551786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42445940-0139-4d82-83a4-e1045c965d35 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:17.558765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:17.560223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:17.560532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:17.560812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:17.576248 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:17.579083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:17.600181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1197/2389] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:42:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:42:17.703612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:17.703612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:17.715499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:17.722446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098c6c0-bb0e-4a59-8bed-bf9a7b905b99 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:17.737046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:17.737715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:17.738021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:17.738308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] PATH_INFO: `/v3/projects/30419ac6bc344c5c86d13e0a2e545b7f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:17.740306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30419ac6bc344c5c86d13e0a2e545b7f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:17.741883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:17.933848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30419ac6bc344c5c86d13e0a2e545b7f', 'request_id': 'req-010fdb56-7812-46aa-98e3-50039d330627'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:17.953327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30419ac6bc344c5c86d13e0a2e545b7f', 'request_id': 'req-010fdb56-7812-46aa-98e3-50039d330627'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:17.967186 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30419ac6bc344c5c86d13e0a2e545b7f', 'request_id': 'req-010fdb56-7812-46aa-98e3-50039d330627'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.122139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:18.125405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:18.147690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] The token cache is being invalidate because project 30419ac6bc344c5c86d13e0a2e545b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:18.148026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.149257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-010fdb56-7812-46aa-98e3-50039d330627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.155012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1193/2390] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:17 2022] DELETE /identity/v3/projects/30419ac6bc344c5c86d13e0a2e545b7f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:18.158867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be29891-90c9-4f3a-a2d9-bc4d6abdf546 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.159558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be29891-90c9-4f3a-a2d9-bc4d6abdf546 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.159883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be29891-90c9-4f3a-a2d9-bc4d6abdf546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.164776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3b49b-2386-4ea7-9e27-7e225819591e service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:18.205903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8be29891-90c9-4f3a-a2d9-bc4d6abdf546 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:18.208654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8be29891-90c9-4f3a-a2d9-bc4d6abdf546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:18.282225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:18.282225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.282225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.282225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] PATH_INFO: `/v3/projects/205a3fd429dd4f56918861c582938551` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.283486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=205a3fd429dd4f56918861c582938551)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:18.285145 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:18.302642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1194/2391] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:18.326034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cde37f-c5ad-4864-b3ce-48b878145cf8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.326603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cde37f-c5ad-4864-b3ce-48b878145cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.326879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cde37f-c5ad-4864-b3ce-48b878145cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.357944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-36cde37f-c5ad-4864-b3ce-48b878145cf8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:18.360903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-36cde37f-c5ad-4864-b3ce-48b878145cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:18.375860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1195/2392] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:18.383032 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cde37f-c5ad-4864-b3ce-48b878145cf8 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:18.394885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cde37f-c5ad-4864-b3ce-48b878145cf8 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:18.400401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:18.401649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.401937 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.402319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.403011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:42:18.403326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:18.405060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:18.425093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1196/2393] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:42:18.435355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:18.461575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1b2ea-4e20-40c9-bd6c-5c008d5e96f5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:18.471444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:18.472805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.474039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.479017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.479017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:18.481612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:18.500811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205a3fd429dd4f56918861c582938551', 'request_id': 'req-1ce048fa-8d99-448b-a7ff-4937f79662d7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.510630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205a3fd429dd4f56918861c582938551', 'request_id': 'req-1ce048fa-8d99-448b-a7ff-4937f79662d7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.673196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205a3fd429dd4f56918861c582938551', 'request_id': 'req-1ce048fa-8d99-448b-a7ff-4937f79662d7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.702765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] The token cache is being invalidate because project 205a3fd429dd4f56918861c582938551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:18.703213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.704361 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1ce048fa-8d99-448b-a7ff-4937f79662d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:18.707093 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1198/2394] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:18 2022] DELETE /identity/v3/projects/205a3fd429dd4f56918861c582938551 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:18.714926 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80df6-ed22-487a-82b0-49a5c44f7fdb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.715480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80df6-ed22-487a-82b0-49a5c44f7fdb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.716031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80df6-ed22-487a-82b0-49a5c44f7fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.766051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1197/2395] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:18 2022] POST /identity/v3/projects => generated 377 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:18.768516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:18.770064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ace80df6-ed22-487a-82b0-49a5c44f7fdb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:18.771457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ace80df6-ed22-487a-82b0-49a5c44f7fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:18.773522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:18.914331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:18.926460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1199/2396] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:18.934401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff156-245a-4bae-84f6-0be0ed6aab9b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.934714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff156-245a-4bae-84f6-0be0ed6aab9b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.934982 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff156-245a-4bae-84f6-0be0ed6aab9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.955681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:18.959073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2c508-f66d-4cdc-b789-d88e2447bc36 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:18.966357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:18.967877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:18.969218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:18.970253 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:18.971350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:18.975031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:18.988986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4f0ff156-245a-4bae-84f6-0be0ed6aab9b None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:18.992415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4f0ff156-245a-4bae-84f6-0be0ed6aab9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:19.006614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1200/2397] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:19.011389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff156-245a-4bae-84f6-0be0ed6aab9b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.026182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff156-245a-4bae-84f6-0be0ed6aab9b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.034766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.036076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.039974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.039974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.039974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:42:19.039974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.041472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.068244 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1201/2398] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:42:19.073223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.089265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9ce92-8d45-403d-a38b-319b11d9d65f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.095585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.096453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.096797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.097224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.097930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.100099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.146231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1198/2399] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:18 2022] POST /identity/v3/users => generated 289 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:19.151589 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.164281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628318-111d-4af9-8424-2217e87bf0a1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.170955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.171658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.172027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.172611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.173294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.175258 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.188216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1199/2400] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:19.193772 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.206167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4e40-8e43-446b-901c-030e84d9f42e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.213373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.214424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.214930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.215366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.216389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.218767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.240362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1200/2401] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:19.245478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.265413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d440160-a692-4ca7-bf8b-97cef7a4ac27 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.273220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1202/2402] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:19 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:19.279336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.279676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afeed27-412c-46ca-87a7-3f432d8a4871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.281812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeed27-412c-46ca-87a7-3f432d8a4871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.282306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeed27-412c-46ca-87a7-3f432d8a4871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.282933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeed27-412c-46ca-87a7-3f432d8a4871 admin admin] PATH_INFO: `/v3/projects/fb32ec01baa6482e8fd4caa3bcbf9d36/users/bcf84559d9cb45969b68f8d189f70e61/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.315008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afeed27-412c-46ca-87a7-3f432d8a4871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb32ec01baa6482e8fd4caa3bcbf9d36, user_id=bcf84559d9cb45969b68f8d189f70e61, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.320879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afeed27-412c-46ca-87a7-3f432d8a4871 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.372996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5906a-85a5-4862-b0f3-5b6a7b58291b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.381722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.382826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.383163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.383594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.384524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.387846 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.476008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1201/2403] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:19 2022] PUT /identity/v3/projects/fb32ec01baa6482e8fd4caa3bcbf9d36/users/bcf84559d9cb45969b68f8d189f70e61/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:19.495579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.496140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.496827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.504168 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1203/2404] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:19 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:19.510174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.549485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ab35b-74fd-4093-babe-c980a38fc409 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.582473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] MFA Rules not processed for user `bcf84559d9cb45969b68f8d189f70e61`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:19.587053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:19.731866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.738179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.745425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.745425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.745425 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.749024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.778282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1204/2405] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:19.785944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.811613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235243a-44c4-4ea1-a808-2ecfea2fdca0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.821344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.823813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.823813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.824685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:19.826296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:19.831089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:19.855355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1205/2406] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:19.865935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:19.895216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de3c45-bfcc-48fc-993d-aa9a25a2459c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:19.920817 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e18bbe-98a1-40a5-bd0f-d2e7c6151fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:19.924223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e18bbe-98a1-40a5-bd0f-d2e7c6151fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:19.924223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e18bbe-98a1-40a5-bd0f-d2e7c6151fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:19.925136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e18bbe-98a1-40a5-bd0f-d2e7c6151fd0 admin admin] PATH_INFO: `/v3/projects/daf5dabd4ffc47908c9f0bd18175f98e/users/7c3d5c579d364e62837db456666c787f/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:20.001380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e18bbe-98a1-40a5-bd0f-d2e7c6151fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf5dabd4ffc47908c9f0bd18175f98e, user_id=7c3d5c579d364e62837db456666c787f, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:20.009410 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e18bbe-98a1-40a5-bd0f-d2e7c6151fd0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:20.105648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1202/2407] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 615 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:20.157629 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:20.160542 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:20.178671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1206/2408] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:19 2022] PUT /identity/v3/projects/daf5dabd4ffc47908c9f0bd18175f98e/users/7c3d5c579d364e62837db456666c787f/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:20.181238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bc86-ad86-4c4d-8921-ddc98aa5178a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:20.181731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bc86-ad86-4c4d-8921-ddc98aa5178a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:20.182664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bc86-ad86-4c4d-8921-ddc98aa5178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:20.250725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1d80bc86-ad86-4c4d-8921-ddc98aa5178a None None] MFA Rules not processed for user `7c3d5c579d364e62837db456666c787f`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:20.253082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1d80bc86-ad86-4c4d-8921-ddc98aa5178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:20.258082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:20.301479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5c140-d030-4e81-a6d4-48b7eae1a6a4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:20.399244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:20.423125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:20.423775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:20.425374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:20.425374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:20.426964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1207/2409] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:20.432961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80bc86-ad86-4c4d-8921-ddc98aa5178a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:20.527856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:20.529363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80bc86-ad86-4c4d-8921-ddc98aa5178a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:20.531279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:20.535788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:20.536570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:20.536949 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:20.537363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:20.538422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:20.542021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:20.557666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1203/2410] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:42:20.701591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1208/2411] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:20 2022] POST /identity/v3/projects => generated 381 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:20.707076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:20.728605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4677d9a-af53-44af-89d0-aecb4693190e service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:20.828424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:20.829363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:20.829814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:20.830301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:20.831308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:20.833899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:20.901182 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:20.903301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:20.925698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caabb58-6117-49b5-9249-3bee9d721a22 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:20.933933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:20.934963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:20.935466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:20.936023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] PATH_INFO: `/v3/users/34b1955a937f4053967cf82cedd87a2e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:20.939599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b1955a937f4053967cf82cedd87a2e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:20.943621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:21.007668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1204/2412] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:20 2022] POST /identity/v3/users => generated 297 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:21.012366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:21.023795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861173b-1a5f-4be0-ab90-8de37beb50e8 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:21.028713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:21.029330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:21.029606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:21.029877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:21.030722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:21.032869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:21.044891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1205/2413] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:21.049443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:21.060192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec99974-648d-4e9b-ab5f-c4d1316edd7f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:21.065197 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:21.065872 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:21.066217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:21.066490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] PATH_INFO: `/v3/projects/0ba1ec52272046a7af0548f4631d77a5/users/6e983b532ed344448a04f9a71aedb197/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:21.084146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba1ec52272046a7af0548f4631d77a5, user_id=6e983b532ed344448a04f9a71aedb197, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:21.087146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:21.198139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1206/2414] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:21 2022] PUT /identity/v3/projects/0ba1ec52272046a7af0548f4631d77a5/users/6e983b532ed344448a04f9a71aedb197/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:21.201413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:21.243337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b1955a937f4053967cf82cedd87a2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:21.244158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b1955a937f4053967cf82cedd87a2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:21.255041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b1955a937f4053967cf82cedd87a2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:21.256233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b1955a937f4053967cf82cedd87a2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:21.260868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:21.281270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e1b20-bef9-45a2-8f86-14e1d3a1dcbf admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:21.287124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:21.287757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:21.288031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:21.288300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:21.288958 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:21.290818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:21.303820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1207/2415] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:21.308915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:21.310765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:21.325230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28190153-7ead-47ac-b13e-b00081e98ede admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:21.330981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683dd37-4a12-41db-bc27-d6a455c0a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:21.331665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683dd37-4a12-41db-bc27-d6a455c0a854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:21.331938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683dd37-4a12-41db-bc27-d6a455c0a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:21.332208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683dd37-4a12-41db-bc27-d6a455c0a854 admin admin] PATH_INFO: `/v3/projects/0ba1ec52272046a7af0548f4631d77a5/users/6e983b532ed344448a04f9a71aedb197/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:21.335078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683dd37-4a12-41db-bc27-d6a455c0a854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba1ec52272046a7af0548f4631d77a5, user_id=6e983b532ed344448a04f9a71aedb197, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:21.340556 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683dd37-4a12-41db-bc27-d6a455c0a854 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:21.361547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1208/2416] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:21 2022] PUT /identity/v3/projects/0ba1ec52272046a7af0548f4631d77a5/users/6e983b532ed344448a04f9a71aedb197/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:21.377449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:21.379368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:21.379368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:21.410388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] MFA Rules not processed for user `6e983b532ed344448a04f9a71aedb197`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:21.411831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b1955a937f4053967cf82cedd87a2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:21.412362 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:21.433136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:21.440479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1209/2417] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:20 2022] DELETE /identity/v3/users/34b1955a937f4053967cf82cedd87a2e => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:21.501270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1209/2418] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:21.518855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:21.521690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:21.647737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785be4e-6b97-48c0-a03b-c692c3246895 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:21.654444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:21.655561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:21.656679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:21.657264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:21.758137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:21.759358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:21.771207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1210/2419] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:21.830648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:21.832381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:21.925661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b8509-cb12-48cd-ab00-925a8d410e65 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:22.004068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:22.004726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:22.004996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:22.005262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] PATH_INFO: `/v3/projects/84e82b68d6da45799b52ce9a6ddaf998` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:22.006776 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e82b68d6da45799b52ce9a6ddaf998)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:22.008832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:22.010836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:22.278488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e82b68d6da45799b52ce9a6ddaf998', 'request_id': 'req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:22.278765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:22.289454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e82b68d6da45799b52ce9a6ddaf998', 'request_id': 'req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:22.292638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baa400-3d24-43bf-8a0b-7c306c6e94a9 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:22.300812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e82b68d6da45799b52ce9a6ddaf998', 'request_id': 'req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:22.354854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f3ac3-9bbf-4cc2-9f4c-469ee768a732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:22.355585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3ac3-9bbf-4cc2-9f4c-469ee768a732 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:22.355914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3ac3-9bbf-4cc2-9f4c-469ee768a732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:22.356225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f3ac3-9bbf-4cc2-9f4c-469ee768a732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:22.358480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-675f3ac3-9bbf-4cc2-9f4c-469ee768a732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:22.432606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f3ac3-9bbf-4cc2-9f4c-469ee768a732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:22.434104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f3ac3-9bbf-4cc2-9f4c-469ee768a732 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:22.445627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1211/2420] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:42:22.453851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] The token cache is being invalidate because project 84e82b68d6da45799b52ce9a6ddaf998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:22.454635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:22.455263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-55bf8f39-d70b-4e4a-a121-8f1ad649e8f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:22.457229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1210/2421] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:21 2022] DELETE /identity/v3/projects/84e82b68d6da45799b52ce9a6ddaf998 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:22.464396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e69152-1691-4032-94cb-0037b3964d30 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:22.464725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e69152-1691-4032-94cb-0037b3964d30 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:22.465040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e69152-1691-4032-94cb-0037b3964d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:22.475551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:22.489980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-54e69152-1691-4032-94cb-0037b3964d30 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:22.491442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-54e69152-1691-4032-94cb-0037b3964d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:22.569934 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1212/2422] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:22.578080 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822f994-950c-40b5-9537-3084ae881fbb None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:22.578410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822f994-950c-40b5-9537-3084ae881fbb None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:22.584452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822f994-950c-40b5-9537-3084ae881fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:22.599516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2822f994-950c-40b5-9537-3084ae881fbb None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:22.600738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2822f994-950c-40b5-9537-3084ae881fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:22.609153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1211/2423] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:22.617027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e69152-1691-4032-94cb-0037b3964d30 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:22.630199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e69152-1691-4032-94cb-0037b3964d30 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:22.635558 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:22.636185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:22.636551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:22.636712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:22.637356 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:42:22.637679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:22.639408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:22.656241 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1213/2424] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:42:22.661224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822f994-950c-40b5-9537-3084ae881fbb None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:22.677584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822f994-950c-40b5-9537-3084ae881fbb None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:22.683907 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:22.684554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:22.684825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:22.686272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:22.687720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:22.690579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:22.860822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1212/2425] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:22 2022] POST /identity/v3/projects => generated 405 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:22.865693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:22.877568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2530c8-cfdc-4742-9e2a-b7d121d142e5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:22.949759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:22.950420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:22.950696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:22.950966 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:22.951608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:22.953710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:23.100745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1214/2426] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:22 2022] POST /identity/v3/users => generated 303 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:23.107726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:23.128046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4bbf7-8baf-4090-a577-43066092dde7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:23.137342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:23.138566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:23.139016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:23.139459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:23.141235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:23.144221 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:23.162525 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1213/2427] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:23.167445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:23.179288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9523511-4751-4115-a732-1c0c926d093e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:23.184576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943b8e1-f6ea-4e8c-9de8-9067cf8b4e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:23.185242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943b8e1-f6ea-4e8c-9de8-9067cf8b4e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:23.185517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943b8e1-f6ea-4e8c-9de8-9067cf8b4e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:23.185790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943b8e1-f6ea-4e8c-9de8-9067cf8b4e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:23.186496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943b8e1-f6ea-4e8c-9de8-9067cf8b4e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:23.188336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943b8e1-f6ea-4e8c-9de8-9067cf8b4e73 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:23.201768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1215/2428] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:23.207492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:23.220695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1270de-f924-451c-a231-3f506bf24b54 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:23.226178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:23.226877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:23.227170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:23.229705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] PATH_INFO: `/v3/projects/4a340581e50d41c29b154e22d36d4fcc/users/2fdd6e411a644336852fcc4d94b902f5/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:23.246239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a340581e50d41c29b154e22d36d4fcc, user_id=2fdd6e411a644336852fcc4d94b902f5, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:23.249243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:23.459979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1214/2429] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:23 2022] PUT /identity/v3/projects/4a340581e50d41c29b154e22d36d4fcc/users/2fdd6e411a644336852fcc4d94b902f5/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:23.474366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96322d4f-45dc-422d-9b8f-aab675adff02 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:23.474698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96322d4f-45dc-422d-9b8f-aab675adff02 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:23.474962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96322d4f-45dc-422d-9b8f-aab675adff02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:23.512861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-96322d4f-45dc-422d-9b8f-aab675adff02 None None] MFA Rules not processed for user `2fdd6e411a644336852fcc4d94b902f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:23.524538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-96322d4f-45dc-422d-9b8f-aab675adff02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:23.586826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:23.591444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1216/2430] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:23 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:23.596722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:23.670323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f594313-cb2d-4f4b-a69d-cf5404d28681 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:23.676813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:23.677434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:23.677710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:23.678042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:23.678682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:23.681768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:23.820969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1215/2431] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:23 2022] POST /identity/v3/projects => generated 403 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:23.825568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96322d4f-45dc-422d-9b8f-aab675adff02 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:23.837608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96322d4f-45dc-422d-9b8f-aab675adff02 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:23.905502 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:23.906971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:23.907370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:23.907719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:23.908438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:23.910476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:24.013693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1217/2432] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:23 2022] POST /identity/v3/users => generated 308 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:24.019099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:24.031405 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc594ce-fb74-4690-8621-5e840c22cc86 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:24.036656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:24.037282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:24.037559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:24.037831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:24.038532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:24.040168 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:24.046622 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:24.052873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1216/2433] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:24.058375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:24.076504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab24a9a-6fdb-4f1e-8496-3d3eab3c929d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:24.085157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:24.086246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:24.086731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:24.087234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] PATH_INFO: `/v3/projects/15f08825c6c84e8b9b8d0a67ba848136/users/7b9ea621e6a442fb8e261ac2a315468a/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:24.108227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f08825c6c84e8b9b8d0a67ba848136, user_id=7b9ea621e6a442fb8e261ac2a315468a, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:24.111493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:24.229776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1218/2434] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:24 2022] PUT /identity/v3/projects/15f08825c6c84e8b9b8d0a67ba848136/users/7b9ea621e6a442fb8e261ac2a315468a/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:24.238554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:24.251759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d4152-6f63-4aff-af15-23262f4786ab admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:24.332911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3284d-9782-4cbb-9198-bc7354ba8053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:24.333738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3284d-9782-4cbb-9198-bc7354ba8053 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:24.334192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3284d-9782-4cbb-9198-bc7354ba8053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:24.334637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3284d-9782-4cbb-9198-bc7354ba8053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:24.335397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3284d-9782-4cbb-9198-bc7354ba8053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:24.337406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3284d-9782-4cbb-9198-bc7354ba8053 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:24.356644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1217/2435] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:24.357979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:24.381266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b21162-559e-4256-ba4d-ded75f5687a2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:24.391338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:24.392418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:24.392735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:24.393016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] PATH_INFO: `/v3/projects/15f08825c6c84e8b9b8d0a67ba848136/users/7b9ea621e6a442fb8e261ac2a315468a/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:24.396406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f08825c6c84e8b9b8d0a67ba848136, user_id=7b9ea621e6a442fb8e261ac2a315468a, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:24.400778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:24.420967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1219/2436] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:24 2022] PUT /identity/v3/projects/15f08825c6c84e8b9b8d0a67ba848136/users/7b9ea621e6a442fb8e261ac2a315468a/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:24.434789 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bf47d-136f-4869-8bea-73ed371a8cd1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:24.435084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bf47d-136f-4869-8bea-73ed371a8cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:24.435340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bf47d-136f-4869-8bea-73ed371a8cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:24.469523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-666bf47d-136f-4869-8bea-73ed371a8cd1 None None] MFA Rules not processed for user `7b9ea621e6a442fb8e261ac2a315468a`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:24.470665 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-666bf47d-136f-4869-8bea-73ed371a8cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:24.586984 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1218/2437] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:24 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:24.600165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:24.602071 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:24.676897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d52187-4643-4822-bda1-584654bc22a1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:24.683137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a293d9e-5f4f-4641-a0fe-b3b0d3a58cbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:24.684204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293d9e-5f4f-4641-a0fe-b3b0d3a58cbd service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:24.684694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293d9e-5f4f-4641-a0fe-b3b0d3a58cbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:24.685181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a293d9e-5f4f-4641-a0fe-b3b0d3a58cbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:24.703544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a293d9e-5f4f-4641-a0fe-b3b0d3a58cbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:24.706504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a293d9e-5f4f-4641-a0fe-b3b0d3a58cbd service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:24.725675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1220/2438] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:42:26.491791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bf47d-136f-4869-8bea-73ed371a8cd1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:26.564637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bf47d-136f-4869-8bea-73ed371a8cd1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:26.569783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:26.570555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:26.570883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:26.571199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] PATH_INFO: `/v3/users/2fdd6e411a644336852fcc4d94b902f5` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:26.572516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fdd6e411a644336852fcc4d94b902f5)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:26.574595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:26.614749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:26.960382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fdd6e411a644336852fcc4d94b902f5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:26.961099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fdd6e411a644336852fcc4d94b902f5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:26.961719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fdd6e411a644336852fcc4d94b902f5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:27.091893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fdd6e411a644336852fcc4d94b902f5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:27.100870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fdd6e411a644336852fcc4d94b902f5'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:27.120591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1219/2439] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:26 2022] DELETE /identity/v3/users/2fdd6e411a644336852fcc4d94b902f5 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:27.125878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:27.126686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:27.126686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:27.149322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:27.150697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:27.209041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:27.222229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1221/2440] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:27.233424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:27.235833 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:27.302060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f16da-a12d-4ec1-8fcf-63a20ecfd166 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:27.306025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:27.306657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:27.306933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:27.307198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:27.317954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:27.319443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:27.330915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1220/2441] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:28.037757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:28.039510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:28.057726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d29d01-552b-4b56-9fad-7e624d65f538 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:28.065592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:28.069925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:28.069925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:28.069925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] PATH_INFO: `/v3/projects/4a340581e50d41c29b154e22d36d4fcc` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:28.069925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a340581e50d41c29b154e22d36d4fcc)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:28.070694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:28.264554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a340581e50d41c29b154e22d36d4fcc', 'request_id': 'req-a0f2f7a7-6992-4969-bc55-9b5ed759a119'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.274352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a340581e50d41c29b154e22d36d4fcc', 'request_id': 'req-a0f2f7a7-6992-4969-bc55-9b5ed759a119'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.406562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a340581e50d41c29b154e22d36d4fcc', 'request_id': 'req-a0f2f7a7-6992-4969-bc55-9b5ed759a119'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.428258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] The token cache is being invalidate because project 4a340581e50d41c29b154e22d36d4fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:28.430944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.430944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.430944 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1222/2442] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:28 2022] DELETE /identity/v3/projects/4a340581e50d41c29b154e22d36d4fcc => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:28.437926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:28.448531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864137d-5abe-4dc8-8323-e72d4e6b7eaf service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:28.526626 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:28.527231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:28.527546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:28.527854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] PATH_INFO: `/v3/users/7b9ea621e6a442fb8e261ac2a315468a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:28.529125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9ea621e6a442fb8e261ac2a315468a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:28.532013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:28.822701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:28.824890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9ea621e6a442fb8e261ac2a315468a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.825700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9ea621e6a442fb8e261ac2a315468a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.826476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9ea621e6a442fb8e261ac2a315468a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.914520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9ea621e6a442fb8e261ac2a315468a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.922907 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5be3edff-fcf1-4334-8c16-bccf815be2e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9ea621e6a442fb8e261ac2a315468a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:28.943534 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1221/2443] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:28 2022] DELETE /identity/v3/users/7b9ea621e6a442fb8e261ac2a315468a => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:29.518360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:29.520319 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:29.542620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2f7a7-6992-4969-bc55-9b5ed759a119 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:29.620835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:29.621581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:29.621985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:29.622357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] PATH_INFO: `/v3/projects/15f08825c6c84e8b9b8d0a67ba848136` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:29.623759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15f08825c6c84e8b9b8d0a67ba848136)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:29.626462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:29.765055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f08825c6c84e8b9b8d0a67ba848136', 'request_id': 'req-505b4251-36ac-415e-a695-79c07a34cb00'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:29.772658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f08825c6c84e8b9b8d0a67ba848136', 'request_id': 'req-505b4251-36ac-415e-a695-79c07a34cb00'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:29.869167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f08825c6c84e8b9b8d0a67ba848136', 'request_id': 'req-505b4251-36ac-415e-a695-79c07a34cb00'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:29.884088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] The token cache is being invalidate because project 15f08825c6c84e8b9b8d0a67ba848136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:29.884479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:29.885195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-505b4251-36ac-415e-a695-79c07a34cb00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:29.887000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1223/2444] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:29 2022] DELETE /identity/v3/projects/15f08825c6c84e8b9b8d0a67ba848136 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:29.893488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19becba5-5cc6-41be-b2c7-fcc9ab20545f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:29.894254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19becba5-5cc6-41be-b2c7-fcc9ab20545f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:29.894254 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19becba5-5cc6-41be-b2c7-fcc9ab20545f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:29.915620 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-19becba5-5cc6-41be-b2c7-fcc9ab20545f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:29.916893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-19becba5-5cc6-41be-b2c7-fcc9ab20545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:29.982285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1222/2445] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:29.990814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2158a7-94ea-4331-aaba-bc1d96cae6e8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:29.991087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2158a7-94ea-4331-aaba-bc1d96cae6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:29.991366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2158a7-94ea-4331-aaba-bc1d96cae6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.013578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7c2158a7-94ea-4331-aaba-bc1d96cae6e8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:30.015399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7c2158a7-94ea-4331-aaba-bc1d96cae6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:30.024312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1224/2446] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:30.028985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19becba5-5cc6-41be-b2c7-fcc9ab20545f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:30.040011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19becba5-5cc6-41be-b2c7-fcc9ab20545f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:30.044838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:30.045459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.045739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.046096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.046932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:42:30.047239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:30.048905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:30.061721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1223/2447] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:42:30.066545 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2158a7-94ea-4331-aaba-bc1d96cae6e8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:30.071344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:30.078541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2158a7-94ea-4331-aaba-bc1d96cae6e8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:30.083239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:30.084032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.084554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.084935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.085808 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:30.088072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:30.221655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1225/2448] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:30 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:30.226683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:30.241232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecbcf6-b93f-44f9-9ab5-95d267492742 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:30.321009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:30.322145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.322676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.323189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.324510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:30.328514 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:30.470704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1224/2449] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:30 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:30.475619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:30.490380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7432ed-a22c-42ca-91aa-5d8ed7cb85fb admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:30.503065 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:30.503707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.503985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.504266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.504942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:30.506879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:30.518913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1226/2450] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:30.531114 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:30.542905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee331a-e8d3-4252-8da6-c1940d7680f3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:30.548949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef6184-0e04-44b1-84b9-f0abc23d4d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:30.549815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef6184-0e04-44b1-84b9-f0abc23d4d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.550237 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef6184-0e04-44b1-84b9-f0abc23d4d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.550610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef6184-0e04-44b1-84b9-f0abc23d4d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.551396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef6184-0e04-44b1-84b9-f0abc23d4d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:30.553249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef6184-0e04-44b1-84b9-f0abc23d4d27 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:30.565854 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1225/2451] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:30.570359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:30.582141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269ab9-c745-4081-bcff-e1dac39550fc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:30.587077 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:30.587772 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.588088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.588349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] PATH_INFO: `/v3/projects/d93aa8b272f244e9b0e81a157acc4d2d/users/95eead9d5a2741af8888878c9e08c6ff/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.607450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93aa8b272f244e9b0e81a157acc4d2d, user_id=95eead9d5a2741af8888878c9e08c6ff, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:30.610433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:30.698463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1227/2452] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:30 2022] PUT /identity/v3/projects/d93aa8b272f244e9b0e81a157acc4d2d/users/95eead9d5a2741af8888878c9e08c6ff/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:30.710827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2a537-22b4-4f77-90a6-b055db6a883a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.711090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2a537-22b4-4f77-90a6-b055db6a883a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.711363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2a537-22b4-4f77-90a6-b055db6a883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.746716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6ed2a537-22b4-4f77-90a6-b055db6a883a None None] MFA Rules not processed for user `95eead9d5a2741af8888878c9e08c6ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:30.746716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6ed2a537-22b4-4f77-90a6-b055db6a883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:30.766418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:30.833076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1226/2453] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:30.837924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:30.909514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a037f-cd38-413a-82ba-6d42922fc16a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:30.914567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:30.915195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:30.915477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:30.915749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:30.916396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:30.918669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:31.077259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1228/2454] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:30 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:31.082488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2a537-22b4-4f77-90a6-b055db6a883a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:31.094455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2a537-22b4-4f77-90a6-b055db6a883a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:31.161982 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:31.162720 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:31.163137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:31.163496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:31.164216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:31.166328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:31.299219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1227/2455] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:31 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:31.303327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:31.314169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bd87e-327d-4f1c-8c06-f9158815f4e5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:31.318881 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:31.319494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:31.319770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:31.320041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:31.320680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:31.322308 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:31.334532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1229/2456] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:31.338784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:31.350202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6c589-a96d-4d9e-a85e-f17869273a29 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:31.355518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:31.356188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:31.356496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:31.356771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] PATH_INFO: `/v3/projects/cd985eda9e8a4709bd6fc40bd521ef0b/users/f46b6b32d3c442ab9b4cfeac7e377cfc/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:31.375365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd985eda9e8a4709bd6fc40bd521ef0b, user_id=f46b6b32d3c442ab9b4cfeac7e377cfc, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:31.378287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:31.481837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1228/2457] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:31 2022] PUT /identity/v3/projects/cd985eda9e8a4709bd6fc40bd521ef0b/users/f46b6b32d3c442ab9b4cfeac7e377cfc/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:31.488048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:31.500867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7a8da-e8ad-40e4-9f9f-ccdc42ad94f5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:31.570372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9d87b-cedd-44d0-bf61-376d7d8ba788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:31.571057 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d87b-cedd-44d0-bf61-376d7d8ba788 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:31.571270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d87b-cedd-44d0-bf61-376d7d8ba788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:31.571537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d87b-cedd-44d0-bf61-376d7d8ba788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:31.572206 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9d87b-cedd-44d0-bf61-376d7d8ba788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:31.573922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9d87b-cedd-44d0-bf61-376d7d8ba788 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:31.591805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1230/2458] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:31.592071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:31.602061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a457a-a1e7-4995-81a3-dec76f367c58 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:31.607144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:31.607810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:31.608084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:31.608354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] PATH_INFO: `/v3/projects/cd985eda9e8a4709bd6fc40bd521ef0b/users/f46b6b32d3c442ab9b4cfeac7e377cfc/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:31.611283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd985eda9e8a4709bd6fc40bd521ef0b, user_id=f46b6b32d3c442ab9b4cfeac7e377cfc, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:31.613970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:31.626998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1229/2459] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:31 2022] PUT /identity/v3/projects/cd985eda9e8a4709bd6fc40bd521ef0b/users/f46b6b32d3c442ab9b4cfeac7e377cfc/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:31.650761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ba978-cdaa-4307-9612-5bb54fc9538e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:31.651411 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ba978-cdaa-4307-9612-5bb54fc9538e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:31.651915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ba978-cdaa-4307-9612-5bb54fc9538e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:31.706335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2a7ba978-cdaa-4307-9612-5bb54fc9538e None None] MFA Rules not processed for user `f46b6b32d3c442ab9b4cfeac7e377cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:31.708440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2a7ba978-cdaa-4307-9612-5bb54fc9538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:31.714734 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:31.788347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1231/2460] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:31.800652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:31.802497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:31.868992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432fd75-df5d-465d-9f69-07224cf6563d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:31.872869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:31.873498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:31.873807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:31.874725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:31.902762 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:31.937871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:31.939416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:31.948933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1230/2461] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:42:39.833580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ba978-cdaa-4307-9612-5bb54fc9538e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:39.845838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ba978-cdaa-4307-9612-5bb54fc9538e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:39.849859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafd3fc-cf86-4a3f-9ff5-4489332a67e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:39.850533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafd3fc-cf86-4a3f-9ff5-4489332a67e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:39.850805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafd3fc-cf86-4a3f-9ff5-4489332a67e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:39.851080 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafd3fc-cf86-4a3f-9ff5-4489332a67e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:39.862479 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafd3fc-cf86-4a3f-9ff5-4489332a67e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:39.863940 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafd3fc-cf86-4a3f-9ff5-4489332a67e4 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:39.875993 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1232/2462] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:41.174676 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:41.176272 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:41.258618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6d06d-eaf5-4cd8-b9b1-43fb80e4b225 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:41.264225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:41.265834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:41.266278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:41.266636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] PATH_INFO: `/v3/users/7c3d5c579d364e62837db456666c787f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:41.268457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3d5c579d364e62837db456666c787f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:41.270697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:41.540721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3d5c579d364e62837db456666c787f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:41.543109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3d5c579d364e62837db456666c787f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:41.543790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3d5c579d364e62837db456666c787f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:41.670337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3d5c579d364e62837db456666c787f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:41.680824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3d5c579d364e62837db456666c787f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:41.700285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1231/2463] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:41 2022] DELETE /identity/v3/users/7c3d5c579d364e62837db456666c787f => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:41.706909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:41.713977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:41.713977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:41.739570 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:41.740868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:41.809656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1233/2464] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:41.823156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:41.832731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:41.887367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:41.903810 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43d064-abcf-4b96-8b98-056c47fc9f80 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:41.912666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:41.913661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:41.914141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:41.914582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:41.932124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:41.934679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:41.951853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1232/2465] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:42.547420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:42.548901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:42.562864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c412c-3bbe-42de-9ed2-ffcf58db730e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:42.577849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:42.578542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:42.578922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:42.579082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] PATH_INFO: `/v3/projects/daf5dabd4ffc47908c9f0bd18175f98e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:42.580310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf5dabd4ffc47908c9f0bd18175f98e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:42.583421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:42.741481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf5dabd4ffc47908c9f0bd18175f98e', 'request_id': 'req-09337ded-634f-4b10-b12a-497adca1d8d2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:42.748346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf5dabd4ffc47908c9f0bd18175f98e', 'request_id': 'req-09337ded-634f-4b10-b12a-497adca1d8d2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:42.851698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf5dabd4ffc47908c9f0bd18175f98e', 'request_id': 'req-09337ded-634f-4b10-b12a-497adca1d8d2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:42.865123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] The token cache is being invalidate because project daf5dabd4ffc47908c9f0bd18175f98e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:42.865419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:42.866143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:42.867687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1234/2466] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:42 2022] DELETE /identity/v3/projects/daf5dabd4ffc47908c9f0bd18175f98e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:42.875137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:42.885887 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da027775-c766-411e-9f97-5e8de570135b service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:42.949268 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:42.949911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:42.950218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:42.950529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] PATH_INFO: `/v3/users/6e983b532ed344448a04f9a71aedb197` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:42.951842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e983b532ed344448a04f9a71aedb197)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:42.954805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:43.198025 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e983b532ed344448a04f9a71aedb197'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:43.199151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e983b532ed344448a04f9a71aedb197'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:43.199905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e983b532ed344448a04f9a71aedb197'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:43.314220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e983b532ed344448a04f9a71aedb197'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:43.324377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-c3041569-7cde-433c-b7a2-c02a69217d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e983b532ed344448a04f9a71aedb197'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:43.351227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1233/2467] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:42 2022] DELETE /identity/v3/users/6e983b532ed344448a04f9a71aedb197 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:43.910953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:43.913538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:43.933888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09337ded-634f-4b10-b12a-497adca1d8d2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:43.946679 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:44.008540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:44.009188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:44.009552 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:44.009832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] PATH_INFO: `/v3/projects/0ba1ec52272046a7af0548f4631d77a5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:44.011036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba1ec52272046a7af0548f4631d77a5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:44.012952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:44.209939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba1ec52272046a7af0548f4631d77a5', 'request_id': 'req-3f643e77-7687-444b-9726-821467ba9d4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:44.221123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba1ec52272046a7af0548f4631d77a5', 'request_id': 'req-3f643e77-7687-444b-9726-821467ba9d4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:44.474862 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba1ec52272046a7af0548f4631d77a5', 'request_id': 'req-3f643e77-7687-444b-9726-821467ba9d4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:44.496262 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] The token cache is being invalidate because project 0ba1ec52272046a7af0548f4631d77a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:44.496580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:44.497296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f643e77-7687-444b-9726-821467ba9d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:44.499117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1235/2468] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:43 2022] DELETE /identity/v3/projects/0ba1ec52272046a7af0548f4631d77a5 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:44.504452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d34ff-4ac8-4cd3-945c-c755b0a2a0e3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:44.505543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d34ff-4ac8-4cd3-945c-c755b0a2a0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:44.505543 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d34ff-4ac8-4cd3-945c-c755b0a2a0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:44.526337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fb6d34ff-4ac8-4cd3-945c-c755b0a2a0e3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:44.527782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fb6d34ff-4ac8-4cd3-945c-c755b0a2a0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:44.600397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1234/2469] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:44.609164 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df45523-194a-4c64-b5de-520a353dfccc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:44.609489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df45523-194a-4c64-b5de-520a353dfccc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:44.609908 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df45523-194a-4c64-b5de-520a353dfccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:44.631650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8df45523-194a-4c64-b5de-520a353dfccc None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:44.632900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8df45523-194a-4c64-b5de-520a353dfccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:44.642113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1236/2470] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:44.647365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d34ff-4ac8-4cd3-945c-c755b0a2a0e3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:44.661745 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d34ff-4ac8-4cd3-945c-c755b0a2a0e3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:44.669914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:44.671450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:44.671954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:44.672518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:44.673763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:42:44.674351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:44.677508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:44.680476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:44.693703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1235/2471] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:42:44.698526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df45523-194a-4c64-b5de-520a353dfccc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:44.709473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df45523-194a-4c64-b5de-520a353dfccc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:44.714615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:44.715288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:44.715678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:44.715996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:44.716689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:44.719700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:44.848744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1237/2472] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:44 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:44.853671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:44.866061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c9dc9-5d56-40ed-ac75-652718d591d3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:44.949385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:44.950048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:44.950391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:44.950659 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:44.951401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:44.954048 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:45.074324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1236/2473] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:44 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:45.079751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:45.096313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d0b41-16ac-4335-aa69-312221cac555 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:45.104246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:45.105852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:45.106582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:45.107247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:45.109041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:45.112352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:45.129460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1238/2474] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:45.134259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:45.145985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38293ef-e5b4-44df-b7a3-8bfc5423d94b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:45.150852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e332a12-0724-418f-b972-78dff1925f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:45.151489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332a12-0724-418f-b972-78dff1925f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:45.151750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332a12-0724-418f-b972-78dff1925f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:45.152016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332a12-0724-418f-b972-78dff1925f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:45.152695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e332a12-0724-418f-b972-78dff1925f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:45.154394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e332a12-0724-418f-b972-78dff1925f6e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:45.166329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1237/2475] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:45.171083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:45.183387 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cc172-87cc-437f-aa9e-20280cdc3906 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:45.190605 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:45.191588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:45.191997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:45.192466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] PATH_INFO: `/v3/projects/a6dc6b9a9ca44311840e563e0227f088/users/df81f09672d645d5bbf7d7754eeb531b/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:45.212165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dc6b9a9ca44311840e563e0227f088, user_id=df81f09672d645d5bbf7d7754eeb531b, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:45.215016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:45.353273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1239/2476] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:45 2022] PUT /identity/v3/projects/a6dc6b9a9ca44311840e563e0227f088/users/df81f09672d645d5bbf7d7754eeb531b/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:45.367828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1b2d-a502-4678-87ca-d1aaec56cec9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:45.368130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1b2d-a502-4678-87ca-d1aaec56cec9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:45.368404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1b2d-a502-4678-87ca-d1aaec56cec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:45.412404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-4aaa1b2d-a502-4678-87ca-d1aaec56cec9 None None] MFA Rules not processed for user `df81f09672d645d5bbf7d7754eeb531b`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:45.413843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4aaa1b2d-a502-4678-87ca-d1aaec56cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:45.521783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1238/2477] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:45.523723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:45.615229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:45.641776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f58e73-c77d-43f2-9803-4d0d1b9236be admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:45.650191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:45.651243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:45.651913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:45.652247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:45.653417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:45.658022 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:45.821447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1240/2478] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:45 2022] POST /identity/v3/projects => generated 387 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:45.826248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1b2d-a502-4678-87ca-d1aaec56cec9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:45.839777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1b2d-a502-4678-87ca-d1aaec56cec9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:45.915117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:45.915653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:45.915933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:45.916203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:45.916830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:45.918826 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:45.928999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:46.035636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1239/2479] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:45 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:46.040186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:46.053617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f244ef1-ec5b-47f0-b1f9-4f7b387735b9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:46.059108 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:46.059771 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:46.060053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:46.060329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:46.061165 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:46.065559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:46.085144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1241/2480] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:46.099485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:46.115118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a51a6a-890f-45ea-a5c7-233306a2fd73 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:46.121374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:46.122263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:46.122646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:46.123088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] PATH_INFO: `/v3/projects/2fd8084b26fd48259008d578e2faad58/users/12d6a161c9c94f64984e07c184451a36/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:46.147913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd8084b26fd48259008d578e2faad58, user_id=12d6a161c9c94f64984e07c184451a36, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:46.151371 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:46.259960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1240/2481] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:46 2022] PUT /identity/v3/projects/2fd8084b26fd48259008d578e2faad58/users/12d6a161c9c94f64984e07c184451a36/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:46.266677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:46.277632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06ac8a-2d28-4e8a-b029-0be532791824 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:46.343802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb09ad-0a17-435e-9d4a-0bfb99d345d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:46.344517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb09ad-0a17-435e-9d4a-0bfb99d345d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:46.344804 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb09ad-0a17-435e-9d4a-0bfb99d345d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:46.345078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb09ad-0a17-435e-9d4a-0bfb99d345d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:46.345735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb09ad-0a17-435e-9d4a-0bfb99d345d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:46.347484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb09ad-0a17-435e-9d4a-0bfb99d345d3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:46.358888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1242/2482] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:46.364179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:46.383834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5274-6554-46aa-803c-c58bd66040c5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:46.391009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:46.391701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:46.391983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:46.392261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] PATH_INFO: `/v3/projects/2fd8084b26fd48259008d578e2faad58/users/12d6a161c9c94f64984e07c184451a36/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:46.394476 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd8084b26fd48259008d578e2faad58, user_id=12d6a161c9c94f64984e07c184451a36, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:46.397143 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:46.409836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1241/2483] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:46 2022] PUT /identity/v3/projects/2fd8084b26fd48259008d578e2faad58/users/12d6a161c9c94f64984e07c184451a36/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:46.421620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490142-108b-4738-8129-4886c2ee51f1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:46.421873 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490142-108b-4738-8129-4886c2ee51f1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:46.422224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490142-108b-4738-8129-4886c2ee51f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:46.455196 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b0490142-108b-4738-8129-4886c2ee51f1 None None] MFA Rules not processed for user `12d6a161c9c94f64984e07c184451a36`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:46.456788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b0490142-108b-4738-8129-4886c2ee51f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:46.537043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1243/2484] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:46.549623 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:46.552356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:46.649768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d47f9-c8f1-4f3f-a098-2ee12ae64fb3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:46.653705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a474-0f5b-4646-a941-fd383d771bca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:46.654365 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a474-0f5b-4646-a941-fd383d771bca service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:46.654637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a474-0f5b-4646-a941-fd383d771bca service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:46.654985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a474-0f5b-4646-a941-fd383d771bca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:46.665707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064a474-0f5b-4646-a941-fd383d771bca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:46.667188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064a474-0f5b-4646-a941-fd383d771bca service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:46.678688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1242/2485] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:46.830526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490142-108b-4738-8129-4886c2ee51f1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:46.884368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:46.918383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490142-108b-4738-8129-4886c2ee51f1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:46.923525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:46.924188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:46.924526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:46.924759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] PATH_INFO: `/v3/users/df81f09672d645d5bbf7d7754eeb531b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:46.925907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df81f09672d645d5bbf7d7754eeb531b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:46.927979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:47.321898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df81f09672d645d5bbf7d7754eeb531b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:47.322609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df81f09672d645d5bbf7d7754eeb531b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:47.330743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df81f09672d645d5bbf7d7754eeb531b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:47.331449 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df81f09672d645d5bbf7d7754eeb531b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:47.502886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df81f09672d645d5bbf7d7754eeb531b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:47.523901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1244/2486] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:46 2022] DELETE /identity/v3/users/df81f09672d645d5bbf7d7754eeb531b => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:47.528454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:47.528738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:47.529015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:47.549172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:47.550472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:47.614209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1243/2487] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:47.625741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:47.627251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:47.729767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607ea2-683f-41ae-88d3-131d63d8d907 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:47.734220 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:47.735027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:47.735368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:47.735638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:47.745890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:47.747365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:47.758665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1245/2488] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:48.385957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:48.388391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:48.409578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f333b-56c4-4f47-a399-6249b2d994b9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:48.414638 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:48.415468 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:48.415818 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:48.416153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] PATH_INFO: `/v3/projects/a6dc6b9a9ca44311840e563e0227f088` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:48.417487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dc6b9a9ca44311840e563e0227f088)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:48.419504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:48.544944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dc6b9a9ca44311840e563e0227f088', 'request_id': 'req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:48.548175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:48.552330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dc6b9a9ca44311840e563e0227f088', 'request_id': 'req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:48.558966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dc6b9a9ca44311840e563e0227f088', 'request_id': 'req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:48.680043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] The token cache is being invalidate because project a6dc6b9a9ca44311840e563e0227f088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:48.680299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:48.680988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:48.682554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1244/2489] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:48 2022] DELETE /identity/v3/projects/a6dc6b9a9ca44311840e563e0227f088 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:48.687229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:48.698132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6917731-ffac-4cf9-a519-ba7ade42aadc service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:48.761338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:48.770868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:48.770868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:48.770868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:48.770868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] PATH_INFO: `/v3/users/12d6a161c9c94f64984e07c184451a36` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:48.772577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d6a161c9c94f64984e07c184451a36)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:48.773777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:49.009027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d6a161c9c94f64984e07c184451a36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:49.009608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d6a161c9c94f64984e07c184451a36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:49.017359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d6a161c9c94f64984e07c184451a36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:49.018114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d6a161c9c94f64984e07c184451a36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:49.085591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d6a161c9c94f64984e07c184451a36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:49.114702 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1246/2490] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:48 2022] DELETE /identity/v3/users/12d6a161c9c94f64984e07c184451a36 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:49.434730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:49.436876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:49.555768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67fdd7-72e3-474a-8ce9-721bfe6a7d92 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:49.622598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91000ebf-6813-40d7-a05d-073273357ce1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:49.623222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000ebf-6813-40d7-a05d-073273357ce1 service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:49.623507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000ebf-6813-40d7-a05d-073273357ce1 service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:49.624646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91000ebf-6813-40d7-a05d-073273357ce1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:49.626632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-91000ebf-6813-40d7-a05d-073273357ce1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:49.640013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91000ebf-6813-40d7-a05d-073273357ce1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:49.642414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91000ebf-6813-40d7-a05d-073273357ce1 service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:49.654235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1245/2491] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:49.761797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:49.764090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:49.863401 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79c51c-b0ba-47f1-aed8-e6e69f3ad0cd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:49.868426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:49.869130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:49.869467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:49.869769 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] PATH_INFO: `/v3/projects/2fd8084b26fd48259008d578e2faad58` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:49.871015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd8084b26fd48259008d578e2faad58)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:49.872991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:50.029382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd8084b26fd48259008d578e2faad58', 'request_id': 'req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:50.035952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd8084b26fd48259008d578e2faad58', 'request_id': 'req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:50.188489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd8084b26fd48259008d578e2faad58', 'request_id': 'req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:50.202689 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] The token cache is being invalidate because project 2fd8084b26fd48259008d578e2faad58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:42:50.203049 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:50.203941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f11c65ab-13f3-4bc5-947b-f06d7eb872e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:42:50.205666 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1247/2492] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:42:49 2022] DELETE /identity/v3/projects/2fd8084b26fd48259008d578e2faad58 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:50.211657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1c1bc-2918-4414-ab51-fd8a1f469c97 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:50.211968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1c1bc-2918-4414-ab51-fd8a1f469c97 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:50.212202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1c1bc-2918-4414-ab51-fd8a1f469c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:50.237389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-59d1c1bc-2918-4414-ab51-fd8a1f469c97 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:50.238739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-59d1c1bc-2918-4414-ab51-fd8a1f469c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:50.316763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:50.343851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1246/2493] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:50.352197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee71f8-5111-4654-9b4c-40449e5b54e3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:50.352534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee71f8-5111-4654-9b4c-40449e5b54e3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:50.352722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee71f8-5111-4654-9b4c-40449e5b54e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:50.363002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:50.382818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dbee71f8-5111-4654-9b4c-40449e5b54e3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:50.384300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dbee71f8-5111-4654-9b4c-40449e5b54e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:50.395683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1248/2494] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:42:50.401112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1c1bc-2918-4414-ab51-fd8a1f469c97 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:50.420166 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1c1bc-2918-4414-ab51-fd8a1f469c97 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:50.428354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:50.429401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:50.429961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:50.430486 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:50.431838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:42:50.432360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:50.435506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:50.458314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1247/2495] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:42:50.463815 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee71f8-5111-4654-9b4c-40449e5b54e3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:50.476002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee71f8-5111-4654-9b4c-40449e5b54e3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:50.481185 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:50.481819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:50.482205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:50.482481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:50.483839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:50.485661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:50.636820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1249/2496] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:50 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:50.638303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:50.650913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845c1b5-e15d-4448-a294-ba5d63d81fa1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:50.722424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:50.722956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:50.723233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:50.723506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:50.724144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:50.726116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:50.892014 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1248/2497] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:50 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:42:50.898199 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:50.917096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdcc37-0b54-497d-a1e9-191dca6cbb00 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:50.926370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:50.927381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:50.927855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:50.928331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:50.929420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:50.932599 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:50.949119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1250/2498] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:42:50.952655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:50.965285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403708c-2eac-457c-8dd4-954d590feddf admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:50.972260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a67f3-2864-4c3a-816a-446ee419eab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:50.973294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a67f3-2864-4c3a-816a-446ee419eab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:50.973558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a67f3-2864-4c3a-816a-446ee419eab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:50.973975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a67f3-2864-4c3a-816a-446ee419eab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:50.974890 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a67f3-2864-4c3a-816a-446ee419eab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:50.976961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a67f3-2864-4c3a-816a-446ee419eab9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:50.991052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1249/2499] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:50.995902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:51.007695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae711-4cd1-46c3-b6f0-9dc63d9441d0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:51.012490 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:51.013174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:51.013450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:51.013724 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] PATH_INFO: `/v3/projects/acb791d25a684852b23e3b8f635cd8ef/users/e1440f3bd04e481889b46f0eb820d0a0/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:51.031810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb791d25a684852b23e3b8f635cd8ef, user_id=e1440f3bd04e481889b46f0eb820d0a0, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:51.034740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:51.139103 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1251/2500] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:50 2022] PUT /identity/v3/projects/acb791d25a684852b23e3b8f635cd8ef/users/e1440f3bd04e481889b46f0eb820d0a0/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:51.153098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fc039-ec16-465b-91aa-6abaca2793fd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:51.153400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fc039-ec16-465b-91aa-6abaca2793fd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:51.153641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fc039-ec16-465b-91aa-6abaca2793fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:51.187591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-077fc039-ec16-465b-91aa-6abaca2793fd None None] MFA Rules not processed for user `e1440f3bd04e481889b46f0eb820d0a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:51.188916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-077fc039-ec16-465b-91aa-6abaca2793fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:51.271102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1250/2501] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:51.276826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:51.369858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84965d7-81f4-4e34-898b-682a90ae1950 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:51.376665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:51.377535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:51.377851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:51.378180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:51.378900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:51.381337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:51.497948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1252/2502] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:42:51 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:51.502867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fc039-ec16-465b-91aa-6abaca2793fd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:51.515245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fc039-ec16-465b-91aa-6abaca2793fd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:51.566332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:51.581334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:51.582037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:51.584948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:51.584948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:51.584948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:51.585357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:51.692184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1251/2503] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:42:51 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:42:51.698782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:51.718423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b604d-e2d4-4fa7-9449-6d1c7430493d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:51.726274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:51.726917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:51.727191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:51.727465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:51.728125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:51.729845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:51.743036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1253/2504] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:51.748881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:51.760643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050dfa0-672d-45f1-869c-db49d609d1cc admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:51.766234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:51.766919 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:51.767194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:51.767467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] PATH_INFO: `/v3/projects/c1d21f095f2a4834989c3454cd362952/users/56a55b5e7ebc4f2bae6af3d85af365b4/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:51.785044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d21f095f2a4834989c3454cd362952, user_id=56a55b5e7ebc4f2bae6af3d85af365b4, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:51.787811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:51.897570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1252/2505] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:51 2022] PUT /identity/v3/projects/c1d21f095f2a4834989c3454cd362952/users/56a55b5e7ebc4f2bae6af3d85af365b4/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:42:51.901837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:51.913929 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874da6-b8a9-4f78-a8a2-5d15bda2f03b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:51.976385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95163ad2-83ce-4388-a3c4-603131a1d6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:51.977507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95163ad2-83ce-4388-a3c4-603131a1d6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:51.978013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95163ad2-83ce-4388-a3c4-603131a1d6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:51.978525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95163ad2-83ce-4388-a3c4-603131a1d6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:51.979698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95163ad2-83ce-4388-a3c4-603131a1d6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:51.982896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95163ad2-83ce-4388-a3c4-603131a1d6f6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:51.998854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1254/2506] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:42:52.003731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:52.016053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217bff-5dab-42d1-92e3-dd9840ec8711 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:52.021230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:52.021905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:52.022208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:52.022481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] PATH_INFO: `/v3/projects/c1d21f095f2a4834989c3454cd362952/users/56a55b5e7ebc4f2bae6af3d85af365b4/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:52.024649 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d21f095f2a4834989c3454cd362952, user_id=56a55b5e7ebc4f2bae6af3d85af365b4, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:52.027467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:52.039602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1253/2507] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:42:52 2022] PUT /identity/v3/projects/c1d21f095f2a4834989c3454cd362952/users/56a55b5e7ebc4f2bae6af3d85af365b4/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:42:52.051370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:52.051655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:52.051932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:52.074210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:42:52.084075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] MFA Rules not processed for user `56a55b5e7ebc4f2bae6af3d85af365b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:42:52.085372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:52.171426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1255/2508] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:42:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:42:52.179698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:52.181792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:52.281158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a77585-3996-401f-9080-325616d93937 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:52.287580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323fa5b-787e-4fcb-a53a-5953ddb8ca06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:52.288633 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323fa5b-787e-4fcb-a53a-5953ddb8ca06 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:52.289117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323fa5b-787e-4fcb-a53a-5953ddb8ca06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:52.289680 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323fa5b-787e-4fcb-a53a-5953ddb8ca06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:52.374710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323fa5b-787e-4fcb-a53a-5953ddb8ca06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:52.376853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323fa5b-787e-4fcb-a53a-5953ddb8ca06 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:42:52.388038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1254/2509] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:42:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:42:59.843841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:42:59.845610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:42:59.938414 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584760f5-a8d3-48a2-8415-7e99a2cc3731 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:42:59.946762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:42:59.947910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:42:59.948390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:42:59.948973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] PATH_INFO: `/v3/users/bcf84559d9cb45969b68f8d189f70e61` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:42:59.951222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf84559d9cb45969b68f8d189f70e61)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:42:59.960408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:00.241973 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf84559d9cb45969b68f8d189f70e61'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:00.242857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf84559d9cb45969b68f8d189f70e61'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:00.251811 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf84559d9cb45969b68f8d189f70e61'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:00.252586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf84559d9cb45969b68f8d189f70e61'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:00.531976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf84559d9cb45969b68f8d189f70e61'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:00.556095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1256/2510] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:42:59 2022] DELETE /identity/v3/users/bcf84559d9cb45969b68f8d189f70e61 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:00.561786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:00.562657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:00.562853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:00.596006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:00.599464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:00.683448 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:00.685662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:00.704981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:00.710604 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:00.730696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1255/2511] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:00.739704 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:00.741771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:00.782782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666cdc5-19df-4a44-84b8-9ccee1a44ee5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:00.790618 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858ac17-6e7f-42b7-903a-45b6f2def129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:00.791697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858ac17-6e7f-42b7-903a-45b6f2def129 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:00.792092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858ac17-6e7f-42b7-903a-45b6f2def129 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:00.792467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858ac17-6e7f-42b7-903a-45b6f2def129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:00.794664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4858ac17-6e7f-42b7-903a-45b6f2def129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:00.862635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4b5b5-5702-40ae-a21a-ed423b0d513c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:00.869292 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:00.870328 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:00.870763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:00.871205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:00.890187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:00.893973 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:00.913093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1256/2512] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:00.914135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858ac17-6e7f-42b7-903a-45b6f2def129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:00.918355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858ac17-6e7f-42b7-903a-45b6f2def129 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:00.936813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1257/2513] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:01.471282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:01.473862 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:01.494928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a1b70-444e-424b-b703-bae4c65a3e8b service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:01.507546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:01.509208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:01.509208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:01.509537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] PATH_INFO: `/v3/projects/fb32ec01baa6482e8fd4caa3bcbf9d36` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:01.511773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb32ec01baa6482e8fd4caa3bcbf9d36)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:01.518226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:01.710642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb32ec01baa6482e8fd4caa3bcbf9d36', 'request_id': 'req-8c14388c-3d7f-47af-a0e5-706edbe9436d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:01.719006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb32ec01baa6482e8fd4caa3bcbf9d36', 'request_id': 'req-8c14388c-3d7f-47af-a0e5-706edbe9436d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:01.726540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb32ec01baa6482e8fd4caa3bcbf9d36', 'request_id': 'req-8c14388c-3d7f-47af-a0e5-706edbe9436d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:01.878645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] The token cache is being invalidate because project fb32ec01baa6482e8fd4caa3bcbf9d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:01.879278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:01.880754 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-8c14388c-3d7f-47af-a0e5-706edbe9436d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:01.888467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1257/2514] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:01 2022] DELETE /identity/v3/projects/fb32ec01baa6482e8fd4caa3bcbf9d36 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:01.891602 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d71d5b-5d90-4b85-ac44-08e71f704d86 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:01.892331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d71d5b-5d90-4b85-ac44-08e71f704d86 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:01.893042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d71d5b-5d90-4b85-ac44-08e71f704d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:01.936384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-44d71d5b-5d90-4b85-ac44-08e71f704d86 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:01.937896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-44d71d5b-5d90-4b85-ac44-08e71f704d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:02.040991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1258/2515] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:02.049417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cefed9-ab0f-4954-83bc-3ddf2ee9c91f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.049784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cefed9-ab0f-4954-83bc-3ddf2ee9c91f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.050096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cefed9-ab0f-4954-83bc-3ddf2ee9c91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.073065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-32cefed9-ab0f-4954-83bc-3ddf2ee9c91f None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:02.074813 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-32cefed9-ab0f-4954-83bc-3ddf2ee9c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:02.084656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1258/2516] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:02.089378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d71d5b-5d90-4b85-ac44-08e71f704d86 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:02.100930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d71d5b-5d90-4b85-ac44-08e71f704d86 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:02.105948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:02.106680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.106963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.107238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.108875 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:02.109298 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:02.110998 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:02.124824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1259/2517] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:43:02.138257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cefed9-ab0f-4954-83bc-3ddf2ee9c91f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:02.148415 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cefed9-ab0f-4954-83bc-3ddf2ee9c91f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:02.161586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:02.162898 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.163401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.163842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.164970 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:02.168639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:02.386167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1259/2518] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:02 2022] POST /identity/v3/projects => generated 373 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:02.391214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:02.401955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd26a2-58e9-492b-8761-f7fef26050e3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:02.463245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:02.474395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:02.475044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.475323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.475594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.476252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:02.478454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:02.579511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1260/2519] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:02 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:02.584471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:02.596847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ffde-5395-4732-a00d-54bb80c79605 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:02.601801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:02.602440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.602851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.603193 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.603967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:02.605567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:02.617105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1260/2520] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:02.622194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:02.634670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c380-a153-4b71-83e1-63e5e5be7ecf admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:02.640589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e3f0a-184a-443d-94eb-29a1a4b28d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:02.641701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e3f0a-184a-443d-94eb-29a1a4b28d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.641701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e3f0a-184a-443d-94eb-29a1a4b28d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.642061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e3f0a-184a-443d-94eb-29a1a4b28d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.642511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e3f0a-184a-443d-94eb-29a1a4b28d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:02.644139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e3f0a-184a-443d-94eb-29a1a4b28d7b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:02.658687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1261/2521] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:02.663241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:02.673948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fa905-1d1b-420b-86f5-064c0bbdfda7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:02.680893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:02.680893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.680893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.680893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] PATH_INFO: `/v3/projects/3cbef7b1e9bc43ba8c85a75d6941af22/users/86aecb5c600544fc81ec2edab12a4c2e/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.696726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cbef7b1e9bc43ba8c85a75d6941af22, user_id=86aecb5c600544fc81ec2edab12a4c2e, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:02.699465 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:02.777645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1261/2522] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:02 2022] PUT /identity/v3/projects/3cbef7b1e9bc43ba8c85a75d6941af22/users/86aecb5c600544fc81ec2edab12a4c2e/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:02.788598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5aa22-5e7c-440f-a58e-1f3710e47d5f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:02.789013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5aa22-5e7c-440f-a58e-1f3710e47d5f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:02.789396 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5aa22-5e7c-440f-a58e-1f3710e47d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:02.822010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a0a5aa22-5e7c-440f-a58e-1f3710e47d5f None None] MFA Rules not processed for user `86aecb5c600544fc81ec2edab12a4c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:02.823335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a0a5aa22-5e7c-440f-a58e-1f3710e47d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:02.926183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1262/2523] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:02.944600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:02.947805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:02.988206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:03.062835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ffb33-60e5-451e-ac04-ee9a547bef02 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:03.069370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d8d3f-121d-44c0-aac6-60c27db72d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:03.070393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d8d3f-121d-44c0-aac6-60c27db72d99 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:03.070880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d8d3f-121d-44c0-aac6-60c27db72d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:03.071367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d8d3f-121d-44c0-aac6-60c27db72d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:03.088532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d8d3f-121d-44c0-aac6-60c27db72d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:03.091001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d8d3f-121d-44c0-aac6-60c27db72d99 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:03.110102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1262/2524] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:09.140638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5aa22-5e7c-440f-a58e-1f3710e47d5f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:09.216712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5aa22-5e7c-440f-a58e-1f3710e47d5f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:09.224265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:09.225106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:09.225388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:09.225684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] PATH_INFO: `/v3/users/86aecb5c600544fc81ec2edab12a4c2e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:09.227090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86aecb5c600544fc81ec2edab12a4c2e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:09.229087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:09.485586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86aecb5c600544fc81ec2edab12a4c2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:09.486559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86aecb5c600544fc81ec2edab12a4c2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:09.500877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86aecb5c600544fc81ec2edab12a4c2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:09.502026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86aecb5c600544fc81ec2edab12a4c2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:09.577117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86aecb5c600544fc81ec2edab12a4c2e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:09.579127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:09.604352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1263/2525] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:09 2022] DELETE /identity/v3/users/86aecb5c600544fc81ec2edab12a4c2e => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:09.609646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:09.614303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:09.614303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:09.632506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:09.633740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:09.746870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1263/2526] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:09.759082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:09.760899 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:09.862754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f850c5-cb52-426a-82d3-c914f5ddc248 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:09.874523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:09.874523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:09.874523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:09.874523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:09.879006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:09.880721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:09.892075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1264/2527] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:10.499301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:10.501066 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:10.518892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33e93a-c2eb-4595-9cf2-8e6df2644d03 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:10.527664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:10.531698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:10.531698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:10.531698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] PATH_INFO: `/v3/projects/3cbef7b1e9bc43ba8c85a75d6941af22` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:10.532506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cbef7b1e9bc43ba8c85a75d6941af22)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:10.536615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:10.763550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cbef7b1e9bc43ba8c85a75d6941af22', 'request_id': 'req-305f7bea-c426-4a92-95f7-73cf5ac3056d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:10.772054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cbef7b1e9bc43ba8c85a75d6941af22', 'request_id': 'req-305f7bea-c426-4a92-95f7-73cf5ac3056d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:10.780219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cbef7b1e9bc43ba8c85a75d6941af22', 'request_id': 'req-305f7bea-c426-4a92-95f7-73cf5ac3056d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:10.826083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:10.830025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:10.852534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0b1a9-6bf2-465e-afb0-6822b477045c service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:10.920951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] The token cache is being invalidate because project 3cbef7b1e9bc43ba8c85a75d6941af22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:10.921313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:10.926503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-305f7bea-c426-4a92-95f7-73cf5ac3056d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:10.926503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1264/2528] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:10 2022] DELETE /identity/v3/projects/3cbef7b1e9bc43ba8c85a75d6941af22 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:10.933399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ece0b9-7328-4d1e-a6db-5d62348d07c2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:10.933866 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ece0b9-7328-4d1e-a6db-5d62348d07c2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:10.934161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ece0b9-7328-4d1e-a6db-5d62348d07c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:10.951880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:10.952777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:10.953051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:10.953481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] PATH_INFO: `/v3/users/e1440f3bd04e481889b46f0eb820d0a0` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:10.954833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1440f3bd04e481889b46f0eb820d0a0)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:10.956827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-90ece0b9-7328-4d1e-a6db-5d62348d07c2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:10.957078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:10.958428 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-90ece0b9-7328-4d1e-a6db-5d62348d07c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:10.969363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1265/2529] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:10.990394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a793022-3d28-4254-8384-93ae848531ce None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:10.990913 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a793022-3d28-4254-8384-93ae848531ce None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:10.991608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a793022-3d28-4254-8384-93ae848531ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.021478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-9a793022-3d28-4254-8384-93ae848531ce None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:11.022950 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9a793022-3d28-4254-8384-93ae848531ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.034601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1266/2530] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:11.041332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a793022-3d28-4254-8384-93ae848531ce None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.059098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a793022-3d28-4254-8384-93ae848531ce None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.070110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.070110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.070110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.070110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.074443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:11.074443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.074443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.098042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1267/2531] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:43:11.102489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.116999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d3a3-16ca-4e6a-8d57-7b0e563ea895 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.124897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.125828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.126326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.126799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.127832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.136018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.169259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:11.235296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1440f3bd04e481889b46f0eb820d0a0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.235983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1440f3bd04e481889b46f0eb820d0a0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.244758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1440f3bd04e481889b46f0eb820d0a0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.245497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1440f3bd04e481889b46f0eb820d0a0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.345812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1268/2532] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:11 2022] POST /identity/v3/projects => generated 371 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:11.345812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.359192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.389923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1440f3bd04e481889b46f0eb820d0a0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.431965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1265/2533] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:10 2022] DELETE /identity/v3/users/e1440f3bd04e481889b46f0eb820d0a0 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:11.434215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.437504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.445547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b954ee6-54c0-48ff-a6cc-d708669efdd5 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.446938 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:11.538925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.539433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.539705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.539981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.542388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.553051 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949b820-2968-40dc-a9a8-6d0ca8402766 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.558317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.559015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.559291 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.559565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] PATH_INFO: `/v3/users/95eead9d5a2741af8888878c9e08c6ff` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.561473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] RBAC: Authorizing `identity:delete_user(user_id=95eead9d5a2741af8888878c9e08c6ff)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.562467 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.563612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.565203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.585131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1269/2534] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:11.586925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.589234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.611824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846380-1cab-4c99-a1df-f94f20f25ae8 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.619900 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8e6e3-2ba1-4dfc-a5c5-f4fea032596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.620952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8e6e3-2ba1-4dfc-a5c5-f4fea032596c admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.621433 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8e6e3-2ba1-4dfc-a5c5-f4fea032596c admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.621835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8e6e3-2ba1-4dfc-a5c5-f4fea032596c admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.622811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8e6e3-2ba1-4dfc-a5c5-f4fea032596c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.626613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8e6e3-2ba1-4dfc-a5c5-f4fea032596c admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.767545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1270/2535] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:11 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:11.768912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736889b7-ada4-4353-be4b-64831eee2c70 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.769194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736889b7-ada4-4353-be4b-64831eee2c70 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.769466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736889b7-ada4-4353-be4b-64831eee2c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.792414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-736889b7-ada4-4353-be4b-64831eee2c70 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:11.794223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-736889b7-ada4-4353-be4b-64831eee2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.811916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1271/2536] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:11.811916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736889b7-ada4-4353-be4b-64831eee2c70 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.820323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736889b7-ada4-4353-be4b-64831eee2c70 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.825492 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.826780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.826780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.826780 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.827459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.829176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.841627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1272/2537] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:11.843262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.845304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.862597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19134a-caf2-42d5-815d-209e02d3e786 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.866621 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.867261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.867539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.867821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.869486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95eead9d5a2741af8888878c9e08c6ff'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.870790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95eead9d5a2741af8888878c9e08c6ff'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.881142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.882765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.886708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95eead9d5a2741af8888878c9e08c6ff'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.887869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95eead9d5a2741af8888878c9e08c6ff'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:11.897489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1273/2538] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:11.898450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.909628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e151c9e-8261-4aca-846f-fe653424960a service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.917839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.919148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.919654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.920775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:11.921947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:11.925304 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:11.946373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1274/2539] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:11.952177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:11.955435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:11.980519 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68ac8-8b89-4ce8-9539-813ea69d9ce7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:11.990412 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:11.991098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:11.991379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:11.991990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] PATH_INFO: `/v3/projects/c2e84676253f422f9459952bfe7b006f/users/c1202f375e89427ba1c20340b5653aa1/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:12.009752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e84676253f422f9459952bfe7b006f, user_id=c1202f375e89427ba1c20340b5653aa1, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:12.013521 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:12.027859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-718ec8ad-7969-4aba-9c7e-94a635d6f77a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95eead9d5a2741af8888878c9e08c6ff'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:12.051412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1266/2540] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:11 2022] DELETE /identity/v3/users/95eead9d5a2741af8888878c9e08c6ff => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:12.060261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc3bc9-1bf6-4bd5-ad0e-82ff196939c7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:12.061274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc3bc9-1bf6-4bd5-ad0e-82ff196939c7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:12.061914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc3bc9-1bf6-4bd5-ad0e-82ff196939c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:12.084031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d0dc3bc9-1bf6-4bd5-ad0e-82ff196939c7 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:12.086193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d0dc3bc9-1bf6-4bd5-ad0e-82ff196939c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:12.157817 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1275/2541] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:11 2022] PUT /identity/v3/projects/c2e84676253f422f9459952bfe7b006f/users/c1202f375e89427ba1c20340b5653aa1/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:12.181120 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1267/2542] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:12.197111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:12.200541 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:12.272696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:12.273180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:12.273662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:12.298055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:12.345763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] MFA Rules not processed for user `c1202f375e89427ba1c20340b5653aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:12.348899 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:12.371071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4fd38-17b9-4624-8532-0bfddbf698ec admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:12.379584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:12.380997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:12.381181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:12.381753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:12.383087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:12.471742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1268/2543] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:12.487595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:12.489077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:12.489284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:12.491731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:12.501889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1276/2544] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:12.565888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed9b61-42d7-496a-b1ee-844470980dfc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:12.570345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aec389ead5164e778a9ee73e7ba7fc50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:12.570996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:12.571272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:12.571542 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:12.583083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:12.584907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:12.596364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1269/2545] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:13.318052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:13.320406 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:13.335827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ac64-b235-4348-8d28-70bb268bcffd service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:13.341285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:13.343269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:13.343561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:13.343839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] PATH_INFO: `/v3/projects/acb791d25a684852b23e3b8f635cd8ef` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:13.345173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb791d25a684852b23e3b8f635cd8ef)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:13.347249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:13.543403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb791d25a684852b23e3b8f635cd8ef', 'request_id': 'req-7f49d583-4103-42d4-8f21-ced7a774700b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:13.554763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb791d25a684852b23e3b8f635cd8ef', 'request_id': 'req-7f49d583-4103-42d4-8f21-ced7a774700b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:13.565071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb791d25a684852b23e3b8f635cd8ef', 'request_id': 'req-7f49d583-4103-42d4-8f21-ced7a774700b'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:13.696673 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] The token cache is being invalidate because project acb791d25a684852b23e3b8f635cd8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:13.697821 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:13.699013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:13.705173 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1277/2546] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:13 2022] DELETE /identity/v3/projects/acb791d25a684852b23e3b8f635cd8ef => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:13.711853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:13.751073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb23a8-a633-4c6a-aae6-ad9fc1242ba5 service glance] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:13.869498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:13.870550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:13.870831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:13.871114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] PATH_INFO: `/v3/users/56a55b5e7ebc4f2bae6af3d85af365b4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:13.872585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a55b5e7ebc4f2bae6af3d85af365b4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:13.875671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:13.957107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:13.957107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:13.996044 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49d583-4103-42d4-8f21-ced7a774700b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:14.021499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:14.023207 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:14.023889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:14.024552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] PATH_INFO: `/v3/projects/d93aa8b272f244e9b0e81a157acc4d2d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:14.027756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93aa8b272f244e9b0e81a157acc4d2d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:14.031841 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:14.157226 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a55b5e7ebc4f2bae6af3d85af365b4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.158601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a55b5e7ebc4f2bae6af3d85af365b4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.173575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a55b5e7ebc4f2bae6af3d85af365b4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.174844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a55b5e7ebc4f2bae6af3d85af365b4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.178162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:14.218373 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93aa8b272f244e9b0e81a157acc4d2d', 'request_id': 'req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.227627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93aa8b272f244e9b0e81a157acc4d2d', 'request_id': 'req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.229172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:14.242244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93aa8b272f244e9b0e81a157acc4d2d', 'request_id': 'req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.291389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a55b5e7ebc4f2bae6af3d85af365b4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.342557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1270/2547] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:13 2022] DELETE /identity/v3/users/56a55b5e7ebc4f2bae6af3d85af365b4 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:14.396744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] The token cache is being invalidate because project d93aa8b272f244e9b0e81a157acc4d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:14.397664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.413403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:14.417209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1278/2548] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:13 2022] DELETE /identity/v3/projects/d93aa8b272f244e9b0e81a157acc4d2d => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:14.427584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:14.434178 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:14.700806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae7f93-3e83-4b7c-a441-9512d6f17b99 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:14.724649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:14.726480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:14.727237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:14.727830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] PATH_INFO: `/v3/users/f46b6b32d3c442ab9b4cfeac7e377cfc` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:14.744372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f46b6b32d3c442ab9b4cfeac7e377cfc)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:14.751176 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:15.127299 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:15.128941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:15.175064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f46b6b32d3c442ab9b4cfeac7e377cfc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.177712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f46b6b32d3c442ab9b4cfeac7e377cfc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.213445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f46b6b32d3c442ab9b4cfeac7e377cfc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.218204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f46b6b32d3c442ab9b4cfeac7e377cfc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.270092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c998-e26f-4199-a9e8-c2d9a428d5b0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:15.276081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:15.277099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:15.277388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:15.277666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] PATH_INFO: `/v3/projects/c1d21f095f2a4834989c3454cd362952` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:15.278924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d21f095f2a4834989c3454cd362952)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:15.281601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:15.345658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9544f432-7fb4-4779-a01c-1d41659582b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f46b6b32d3c442ab9b4cfeac7e377cfc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.377355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1271/2549] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:14 2022] DELETE /identity/v3/users/f46b6b32d3c442ab9b4cfeac7e377cfc => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:15.490404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d21f095f2a4834989c3454cd362952', 'request_id': 'req-981bc979-006d-491f-8230-793fd1da5525'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.499470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d21f095f2a4834989c3454cd362952', 'request_id': 'req-981bc979-006d-491f-8230-793fd1da5525'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.508185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d21f095f2a4834989c3454cd362952', 'request_id': 'req-981bc979-006d-491f-8230-793fd1da5525'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.652341 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] The token cache is being invalidate because project c1d21f095f2a4834989c3454cd362952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:15.652594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.653226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-981bc979-006d-491f-8230-793fd1da5525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:15.655181 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1279/2550] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:15 2022] DELETE /identity/v3/projects/c1d21f095f2a4834989c3454cd362952 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:15.662430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555da79-a00f-4165-a6cd-d5bff57577dd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:15.662861 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555da79-a00f-4165-a6cd-d5bff57577dd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:15.663242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555da79-a00f-4165-a6cd-d5bff57577dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:15.698134 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2555da79-a00f-4165-a6cd-d5bff57577dd None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:15.700699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2555da79-a00f-4165-a6cd-d5bff57577dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:15.806286 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:15.853879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1272/2551] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:15.856845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c390-498e-4cb1-a1ae-05858d7e8ec5 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:15.857610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c390-498e-4cb1-a1ae-05858d7e8ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:15.858266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449c390-498e-4cb1-a1ae-05858d7e8ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:15.903683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3449c390-498e-4cb1-a1ae-05858d7e8ec5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:15.907798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3449c390-498e-4cb1-a1ae-05858d7e8ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:15.935321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1280/2552] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:15.939767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555da79-a00f-4165-a6cd-d5bff57577dd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:15.956450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555da79-a00f-4165-a6cd-d5bff57577dd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:15.966959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:15.967906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:15.968356 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:15.968850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:15.969844 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:15.970714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:15.976193 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.009482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1273/2553] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:43:16.024340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449c390-498e-4cb1-a1ae-05858d7e8ec5 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.052857 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449c390-498e-4cb1-a1ae-05858d7e8ec5 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:16.064904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:16.066026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.066515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.067057 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.068594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:16.084851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.171530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.174898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:16.199879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378caab7-45cb-4e4f-aa7b-74e4b5ddc137 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:16.203045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1281/2554] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:16 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:16.212016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.331669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:16.368999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cf85b-1c82-49b1-bac7-f21e1292c345 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:16.369953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:16.371007 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.371453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.371895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] PATH_INFO: `/v3/projects/cd985eda9e8a4709bd6fc40bd521ef0b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.372155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:16.373338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.373925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.374496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.374714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd985eda9e8a4709bd6fc40bd521ef0b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:16.377240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.377791 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:16.388139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.552974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd985eda9e8a4709bd6fc40bd521ef0b', 'request_id': 'req-007352e8-b182-4984-b7ac-16720c253a51'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:16.562586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd985eda9e8a4709bd6fc40bd521ef0b', 'request_id': 'req-007352e8-b182-4984-b7ac-16720c253a51'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:16.567998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1282/2555] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:16 2022] POST /identity/v3/users => generated 293 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:16.570677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.589056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbd465-ef17-4f6a-be52-a77577a54407 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:16.678839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:16.679478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.679765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.680038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.680711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:16.682420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.684876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd985eda9e8a4709bd6fc40bd521ef0b', 'request_id': 'req-007352e8-b182-4984-b7ac-16720c253a51'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:16.702076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1283/2556] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:16.702076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.702561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] The token cache is being invalidate because project cd985eda9e8a4709bd6fc40bd521ef0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:16.702561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:16.702561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-007352e8-b182-4984-b7ac-16720c253a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:16.704101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1274/2557] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:16 2022] DELETE /identity/v3/projects/cd985eda9e8a4709bd6fc40bd521ef0b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:16.709613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48c1d6-8020-4010-a2c3-e09fb0149c30 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.709918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48c1d6-8020-4010-a2c3-e09fb0149c30 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.710198 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48c1d6-8020-4010-a2c3-e09fb0149c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.712903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd389b-19b7-4619-b52c-f6069086ebd9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:16.718619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:16.719105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.719409 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.719719 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.720309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:16.722011 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.734584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cf48c1d6-8020-4010-a2c3-e09fb0149c30 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:16.735472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1284/2558] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:16.735866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cf48c1d6-8020-4010-a2c3-e09fb0149c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:16.743700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.746653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:16.749773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1275/2559] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:16.768725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b9c9d-5958-4e9d-93cc-45a2fb6e4aa5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.768725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b9c9d-5958-4e9d-93cc-45a2fb6e4aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.769205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b9c9d-5958-4e9d-93cc-45a2fb6e4aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.781177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea17dc-01c5-4d28-873b-94114ccb6457 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:16.809153 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9e366-b23e-4294-9fb9-8d8c078d7193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:16.811609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e366-b23e-4294-9fb9-8d8c078d7193 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.812110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e366-b23e-4294-9fb9-8d8c078d7193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.826363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e366-b23e-4294-9fb9-8d8c078d7193 admin admin] PATH_INFO: `/v3/projects/547369ecc41645b4822b0ddbd4cd3bd2/users/18085b59d9614ca198de336505be931d/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.829307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e40b9c9d-5958-4e9d-93cc-45a2fb6e4aa5 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:16.840959 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e40b9c9d-5958-4e9d-93cc-45a2fb6e4aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:16.895755 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e366-b23e-4294-9fb9-8d8c078d7193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547369ecc41645b4822b0ddbd4cd3bd2, user_id=18085b59d9614ca198de336505be931d, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:16.898598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e366-b23e-4294-9fb9-8d8c078d7193 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.899483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1276/2560] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:16.901284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b9c9d-5958-4e9d-93cc-45a2fb6e4aa5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.932092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b9c9d-5958-4e9d-93cc-45a2fb6e4aa5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:16.944935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:16.946653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:16.946653 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:16.947109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:16.948330 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:16.950408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:16.953284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:16.977704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1277/2561] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:43:16.981642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1285/2562] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:16 2022] PUT /identity/v3/projects/547369ecc41645b4822b0ddbd4cd3bd2/users/18085b59d9614ca198de336505be931d/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:16.983957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:16.995566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ccf33-82df-41d0-996f-a48b5e9198cd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.006154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6643d3-8572-4a81-8e89-24a8dbeb5429 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.007061 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6643d3-8572-4a81-8e89-24a8dbeb5429 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.007410 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6643d3-8572-4a81-8e89-24a8dbeb5429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.078325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-8d6643d3-8572-4a81-8e89-24a8dbeb5429 None None] MFA Rules not processed for user `18085b59d9614ca198de336505be931d`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:17.078325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8d6643d3-8572-4a81-8e89-24a8dbeb5429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:17.125646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:17.168383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.170549 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.171426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.172301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.174349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.179184 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.232461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1286/2563] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:17.239470 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6643d3-8572-4a81-8e89-24a8dbeb5429 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.253076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6643d3-8572-4a81-8e89-24a8dbeb5429 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.261839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.261839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.261839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.261839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.262800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.264076 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.334768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1278/2564] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:16 2022] POST /identity/v3/projects => generated 385 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:17.339370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.382058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1287/2565] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:17 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:17.385432 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.401015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:17.503083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4655d-07c5-4e65-aae5-60f6c4462a0c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.504505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16468e3-f5e9-4c00-9d00-fdf22eb02c96 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.531634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.532792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.533340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.534277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.535624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.535858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.538315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.539247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.539681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.539972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.541430 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.548594 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.757295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1288/2566] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:17 2022] POST /identity/v3/users => generated 292 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:17.766881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.813875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9683b-bb27-4b0b-88ed-309abc5a5377 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.814131 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1279/2567] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:17 2022] POST /identity/v3/users => generated 293 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:17.818965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.819830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.821394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.825861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.825861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.825861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.828778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.837826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a946e7-7978-43ed-841f-ac50b1fad496 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.851546 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.852560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.855189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.855189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.855189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.855680 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1289/2568] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:17.859068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.861804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.876712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f66da-f516-434d-8019-f662dd205dfb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.880023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1280/2569] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:17.881714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.884642 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.885515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.885873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.886234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.886932 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.888935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.899851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611f902-c6d7-4c90-83d7-688baf671600 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.907721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1290/2570] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:17.912540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.912540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.912540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.912540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.913209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.915993 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.916615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.928868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0498d-d609-4670-9964-1170defb5854 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.938183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1281/2571] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:17.938408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb8c87-db59-4556-a9d3-1425a15971e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.938408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb8c87-db59-4556-a9d3-1425a15971e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.938408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb8c87-db59-4556-a9d3-1425a15971e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.938408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb8c87-db59-4556-a9d3-1425a15971e0 admin admin] PATH_INFO: `/v3/projects/129b7e8fc5244ec3b7078e80e745f3a9/users/1f3895eb010e41daac2d9f80456d8810/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.945354 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:17.961096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c069ddd-625f-4216-9cf3-dfea04c4b0f3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:17.970109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb8c87-db59-4556-a9d3-1425a15971e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129b7e8fc5244ec3b7078e80e745f3a9, user_id=1f3895eb010e41daac2d9f80456d8810, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.973377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb8c87-db59-4556-a9d3-1425a15971e0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:17.974583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f642ac-5329-42fa-8946-d7da3b021d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:17.975309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f642ac-5329-42fa-8946-d7da3b021d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:17.975571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f642ac-5329-42fa-8946-d7da3b021d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:17.975842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f642ac-5329-42fa-8946-d7da3b021d46 admin admin] PATH_INFO: `/v3/projects/6d0d9e42b2284c4db2114b96a41aa067/users/fc556b0dd354449fa200dd42eee9c22d/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:17.996060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f642ac-5329-42fa-8946-d7da3b021d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0d9e42b2284c4db2114b96a41aa067, user_id=fc556b0dd354449fa200dd42eee9c22d, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:17.999826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f642ac-5329-42fa-8946-d7da3b021d46 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:18.083218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1291/2572] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:17 2022] PUT /identity/v3/projects/129b7e8fc5244ec3b7078e80e745f3a9/users/1f3895eb010e41daac2d9f80456d8810/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:18.096162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5311-7f28-4fab-8a54-9951037fefda None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:18.096644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5311-7f28-4fab-8a54-9951037fefda None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:18.098105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5311-7f28-4fab-8a54-9951037fefda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:18.126920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1282/2573] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:17 2022] PUT /identity/v3/projects/6d0d9e42b2284c4db2114b96a41aa067/users/fc556b0dd354449fa200dd42eee9c22d/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:18.145156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae0b24-a06a-4b61-9356-8c49b624b562 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:18.145895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae0b24-a06a-4b61-9356-8c49b624b562 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:18.146364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae0b24-a06a-4b61-9356-8c49b624b562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:18.164663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1a8a5311-7f28-4fab-8a54-9951037fefda None None] MFA Rules not processed for user `1f3895eb010e41daac2d9f80456d8810`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:18.166493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1a8a5311-7f28-4fab-8a54-9951037fefda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:18.204943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e7ae0b24-a06a-4b61-9356-8c49b624b562 None None] MFA Rules not processed for user `fc556b0dd354449fa200dd42eee9c22d`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:18.208085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e7ae0b24-a06a-4b61-9356-8c49b624b562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:18.330508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1292/2574] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:18.338510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a5311-7f28-4fab-8a54-9951037fefda None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:18.361961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1283/2575] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:18.369258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae0b24-a06a-4b61-9356-8c49b624b562 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:18.375459 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:18.426660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:18.451670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a5311-7f28-4fab-8a54-9951037fefda None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:18.458060 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:18.458770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:18.459176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:18.459564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:18.460293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:18.462421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:18.496845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae0b24-a06a-4b61-9356-8c49b624b562 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:18.504208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:18.505242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:18.505620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:18.506107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:18.507115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:18.509342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:18.624951 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1293/2576] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:18 2022] POST /identity/v3/projects => generated 383 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:18.626652 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:18.629658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:18.641109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1284/2577] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:18 2022] POST /identity/v3/projects => generated 387 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:18.642619 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:18.756731 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d5671-a883-4d4e-a5d9-ab8fe8dda997 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:18.790285 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582c8db-b904-45af-b6e6-87404ffa0b80 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:18.804201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:18.805041 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:18.805424 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:18.805831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:18.807015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:18.809813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:18.863393 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:18.864176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:18.864497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:18.864814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] PATH_INFO: `/v3/users/c888090dd9f246a79c6be54faf480ed9` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:18.866151 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c888090dd9f246a79c6be54faf480ed9)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:18.868192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:18.925379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1285/2578] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:18 2022] POST /identity/v3/users => generated 298 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:18.926790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:18.946412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1feb9-c397-458c-99fd-674399d83c08 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:18.954712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:18.955730 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:18.956135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:18.956526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:18.957645 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:18.961281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.102829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1286/2579] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:18 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:19.103719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.115497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aef462-d514-42fa-bb1f-a89bfcc93af3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.121254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.121963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.122435 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.123437 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.124776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.126714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.143365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1287/2580] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:19.145716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.157956 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c888090dd9f246a79c6be54faf480ed9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:19.158988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c888090dd9f246a79c6be54faf480ed9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:19.159752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c888090dd9f246a79c6be54faf480ed9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:19.161580 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a8368-4a93-4b9f-aa02-864eab3466a1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.167836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.168627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.169082 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.169447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.171247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.173154 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.184809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1288/2581] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:19.185787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.187647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:19.201766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a358bf-5bff-4989-bbf2-5d6a1321d053 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.208758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.209889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.210415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.210856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] PATH_INFO: `/v3/projects/6bec9455cd734c5da56798503cf66c35/users/eaad53e142474c31824fbe766b349fda/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.239293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bec9455cd734c5da56798503cf66c35, user_id=eaad53e142474c31824fbe766b349fda, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.242305 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.320352 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c888090dd9f246a79c6be54faf480ed9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:19.332713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c888090dd9f246a79c6be54faf480ed9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:19.352449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:19.375978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1289/2582] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:19 2022] PUT /identity/v3/projects/6bec9455cd734c5da56798503cf66c35/users/eaad53e142474c31824fbe766b349fda/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:19.379643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.382304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:19.390201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1294/2583] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:18 2022] DELETE /identity/v3/users/c888090dd9f246a79c6be54faf480ed9 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:19.391214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.414032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:19.515480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76505c-163e-4dca-bf6b-c745e1a15602 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.525805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.527040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.527588 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.528114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] PATH_INFO: `/v3/projects/2878504308be4564bddac45d36e8f198/users/0f61d8a4640b4fc68c0eff9e148d3eb4/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.529750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d28b2-064d-47bd-8b80-b3a514ad4eb9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.541180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.542565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.543055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.543450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.544548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.547213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.564951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2878504308be4564bddac45d36e8f198, user_id=0f61d8a4640b4fc68c0eff9e148d3eb4, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.565344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1295/2584] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:19.565617 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.566777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.587648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdd7cd-8f31-4b26-8827-d1ad968697af admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.601735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0b8e4-4776-41b4-8112-3bdd966892a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.602948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b8e4-4776-41b4-8112-3bdd966892a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.603705 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b8e4-4776-41b4-8112-3bdd966892a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.603893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b8e4-4776-41b4-8112-3bdd966892a1 admin admin] PATH_INFO: `/v3/projects/6bec9455cd734c5da56798503cf66c35/users/eaad53e142474c31824fbe766b349fda/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.607905 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0b8e4-4776-41b4-8112-3bdd966892a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bec9455cd734c5da56798503cf66c35, user_id=eaad53e142474c31824fbe766b349fda, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.613528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0b8e4-4776-41b4-8112-3bdd966892a1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.631967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1296/2585] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:19 2022] PUT /identity/v3/projects/6bec9455cd734c5da56798503cf66c35/users/eaad53e142474c31824fbe766b349fda/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:19.648031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1290/2586] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:19 2022] PUT /identity/v3/projects/2878504308be4564bddac45d36e8f198/users/0f61d8a4640b4fc68c0eff9e148d3eb4/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:19.652438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.652438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.652891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.655705 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.705307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] MFA Rules not processed for user `eaad53e142474c31824fbe766b349fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:19.707283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:19.774974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb492f-3c75-49f6-991f-bd6ed6798090 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.782139 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.783035 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.783377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.783753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.784579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.786840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.801001 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1291/2587] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:19.808075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.829943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f96e9-39c9-49fb-aba0-2c91a6cfc790 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.834344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1297/2588] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:19.834344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.834344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:19.844325 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.845681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.846300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.846855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] PATH_INFO: `/v3/projects/2878504308be4564bddac45d36e8f198/users/0f61d8a4640b4fc68c0eff9e148d3eb4/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.851921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2878504308be4564bddac45d36e8f198, user_id=0f61d8a4640b4fc68c0eff9e148d3eb4, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.855801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.856763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a033eb5-2925-4703-84b6-b28c0e9b9ca7 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.874989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.876101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.876101 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.876847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] PATH_INFO: `/v3/projects/5981da6161ac4717a5b433c31188da53` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.879276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5981da6161ac4717a5b433c31188da53)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:19.891363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1292/2589] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:19 2022] PUT /identity/v3/projects/2878504308be4564bddac45d36e8f198/users/0f61d8a4640b4fc68c0eff9e148d3eb4/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:19.891363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:19.892655 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:19.897333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:19.983773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a4b2-f56e-454a-a0fc-67d133b63efc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:19.988338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f41b3b-9e3c-4922-a590-2f972b911b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:19.989097 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f41b3b-9e3c-4922-a590-2f972b911b50 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:19.989476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f41b3b-9e3c-4922-a590-2f972b911b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:19.989848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f41b3b-9e3c-4922-a590-2f972b911b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:19.991960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e2f41b3b-9e3c-4922-a590-2f972b911b50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:20.038217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:20.054786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5981da6161ac4717a5b433c31188da53', 'request_id': 'req-dad32353-8d80-49a8-b8ef-2e9bfa22683a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.059914 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:20.063961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5981da6161ac4717a5b433c31188da53', 'request_id': 'req-dad32353-8d80-49a8-b8ef-2e9bfa22683a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.072227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5981da6161ac4717a5b433c31188da53', 'request_id': 'req-dad32353-8d80-49a8-b8ef-2e9bfa22683a'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.074099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f41b3b-9e3c-4922-a590-2f972b911b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:20.075581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f41b3b-9e3c-4922-a590-2f972b911b50 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:20.086566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1293/2590] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:20.087972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113964-94b9-482c-8db2-757a61144337 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.088263 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113964-94b9-482c-8db2-757a61144337 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.088632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113964-94b9-482c-8db2-757a61144337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.121890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6c113964-94b9-482c-8db2-757a61144337 None None] MFA Rules not processed for user `0f61d8a4640b4fc68c0eff9e148d3eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:20.124036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6c113964-94b9-482c-8db2-757a61144337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:20.165779 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] The token cache is being invalidate because project 5981da6161ac4717a5b433c31188da53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:20.166452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.167411 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dad32353-8d80-49a8-b8ef-2e9bfa22683a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.171309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1298/2591] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:19 2022] DELETE /identity/v3/projects/5981da6161ac4717a5b433c31188da53 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:20.179255 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c0b7-d289-409e-8786-062b80054c24 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.179760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c0b7-d289-409e-8786-062b80054c24 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.180335 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c0b7-d289-409e-8786-062b80054c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.220119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-6aa8c0b7-d289-409e-8786-062b80054c24 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:20.223259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6aa8c0b7-d289-409e-8786-062b80054c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:20.249885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1294/2592] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:20.268233 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113964-94b9-482c-8db2-757a61144337 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:20.270911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6c113964-94b9-482c-8db2-757a61144337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:20.350766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1299/2593] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:20.361162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bbb5a-e36d-4ab2-a487-600ebb782d35 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.362743 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bbb5a-e36d-4ab2-a487-600ebb782d35 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.364336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bbb5a-e36d-4ab2-a487-600ebb782d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.396271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113964-94b9-482c-8db2-757a61144337 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:20.400926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-681bbb5a-e36d-4ab2-a487-600ebb782d35 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:20.405265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-681bbb5a-e36d-4ab2-a487-600ebb782d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:20.405481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:20.405481 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.405706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.406084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.413770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1300/2594] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:20.419472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bbb5a-e36d-4ab2-a487-600ebb782d35 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:20.428741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:20.431655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:20.433271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bbb5a-e36d-4ab2-a487-600ebb782d35 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:20.440071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:20.442330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.443096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.443444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.445248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:20.445795 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:20.449378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:20.450741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1295/2595] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:20.476981 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1301/2596] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:43:20.489943 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:20.493010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:20.495915 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:20.504964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854046a2-9a52-48aa-a8cd-e76a8d506f23 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:20.510694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:20.511637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.511977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.512313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.513141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:20.516094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:20.523766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b73f3-b68c-4057-801f-34d75cd71b87 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:20.532979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:20.534591 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.538053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.538053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] PATH_INFO: `/v3/users/fc556b0dd354449fa200dd42eee9c22d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.545446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc556b0dd354449fa200dd42eee9c22d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:20.546021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:20.743668 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1296/2597] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:20 2022] POST /identity/v3/projects => generated 373 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:20.749644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:20.819751 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:20.845661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300f5b4-04d4-4bbe-b07d-40618240d902 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:20.854270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:20.854270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:20.854270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:20.854270 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:20.854970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:20.855357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:20.944225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc556b0dd354449fa200dd42eee9c22d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.945480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc556b0dd354449fa200dd42eee9c22d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.946639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc556b0dd354449fa200dd42eee9c22d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:20.991952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1297/2598] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:20 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:20.993367 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:20.995480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:21.014536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27956056-79eb-467f-8896-78d7521ca647 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:21.022780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:21.023783 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.024266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.024738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.025945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:21.030328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:21.051242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1298/2599] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:21.056351 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:21.068950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779d8ba-582b-4695-9c8d-31c1b2494cba admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:21.073489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc556b0dd354449fa200dd42eee9c22d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:21.074526 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:21.075236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.075519 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.075800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.077018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:21.078700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:21.086709 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-3fd1f459-3786-4572-928b-c275c9b0c814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc556b0dd354449fa200dd42eee9c22d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:21.098727 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1299/2600] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:21.104258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:21.118489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1302/2601] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:20 2022] DELETE /identity/v3/users/fc556b0dd354449fa200dd42eee9c22d => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:21.121941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b51120-5220-4565-b9d6-e2274c4b826f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:21.127567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.128208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.128453 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.175503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:21.177289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:21.235359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469837b-d897-4f24-a8c7-3b372dc1bc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:21.235802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:21.236130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469837b-d897-4f24-a8c7-3b372dc1bc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.236482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469837b-d897-4f24-a8c7-3b372dc1bc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.237142 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469837b-d897-4f24-a8c7-3b372dc1bc09 admin admin] PATH_INFO: `/v3/projects/7b7d4eb5c5d8433fa69a554c7abba6ab/users/1035be8bc7a949e2a49f5a77f5ac61cf/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.270598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469837b-d897-4f24-a8c7-3b372dc1bc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7d4eb5c5d8433fa69a554c7abba6ab, user_id=1035be8bc7a949e2a49f5a77f5ac61cf, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:21.276848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469837b-d897-4f24-a8c7-3b372dc1bc09 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:21.318555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1303/2602] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:21.332874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:21.336146 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:21.390076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1300/2603] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:21 2022] PUT /identity/v3/projects/7b7d4eb5c5d8433fa69a554c7abba6ab/users/1035be8bc7a949e2a49f5a77f5ac61cf/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:21.409342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb8880-75c8-4954-9cfd-997a3f41034d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.409663 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb8880-75c8-4954-9cfd-997a3f41034d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.410104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb8880-75c8-4954-9cfd-997a3f41034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.454762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-98bb8880-75c8-4954-9cfd-997a3f41034d None None] MFA Rules not processed for user `1035be8bc7a949e2a49f5a77f5ac61cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:21.455426 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22e6c7-f889-4e4b-9b2c-b286df978095 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:21.456295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-98bb8880-75c8-4954-9cfd-997a3f41034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:21.545246 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:21.547967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.548483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.548968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.568266 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1301/2604] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:21.574026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb8880-75c8-4954-9cfd-997a3f41034d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:21.616525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:21.673123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:21.680053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:21.697330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1304/2605] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:21.702421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb8880-75c8-4954-9cfd-997a3f41034d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:21.710535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:21.711375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.711895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.712598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.713595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:21.716723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:21.832324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1302/2606] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:21 2022] POST /identity/v3/projects => generated 373 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:21.837539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:21.850832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17840fe-c0e7-4257-8ffa-571832ca6bf5 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:21.911323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:21.941435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:21.942753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:21.943229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:21.943546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:21.944700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:21.948276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.058574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1305/2607] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:21 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:22.062957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.074480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3b3b-10de-48b5-a17b-1a3d180b72e3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:22.080953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:22.081743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.082038 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.082389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.083030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:22.085968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.103174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1303/2608] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:22.114577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.126596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9f12c-d4ee-412a-bbfb-62a20950efc3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:22.132346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:22.133113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.133461 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.133839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] PATH_INFO: `/v3/projects/f122fe2e828d4f939c1ffb329b70c239/users/5ecf899820154194a9ec0e9aeb6b56d4/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.170459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f122fe2e828d4f939c1ffb329b70c239, user_id=5ecf899820154194a9ec0e9aeb6b56d4, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:22.173825 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.228682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.230995 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:22.250551 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa2b6-c705-4d2c-8828-8e74289e4e6c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:22.270378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1306/2609] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:22 2022] PUT /identity/v3/projects/f122fe2e828d4f939c1ffb329b70c239/users/5ecf899820154194a9ec0e9aeb6b56d4/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:22.276888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.386161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:22.386740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.387048 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.387307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] PATH_INFO: `/v3/projects/6d0d9e42b2284c4db2114b96a41aa067` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.389253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0d9e42b2284c4db2114b96a41aa067)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:22.393128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.411844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed21412-446b-4ffc-a1c1-96f39901b5e1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:22.420200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:22.421022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.421394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.421771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.422625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:22.424891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.445497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1307/2610] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:22.450356 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.464657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a849a9-4734-4876-a1d7-da60a4c906b0 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:22.470355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af345a52-1363-4f18-b181-60243bd8ac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:22.471053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345a52-1363-4f18-b181-60243bd8ac89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.471340 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345a52-1363-4f18-b181-60243bd8ac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.471657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345a52-1363-4f18-b181-60243bd8ac89 admin admin] PATH_INFO: `/v3/projects/f122fe2e828d4f939c1ffb329b70c239/users/5ecf899820154194a9ec0e9aeb6b56d4/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.473769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af345a52-1363-4f18-b181-60243bd8ac89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f122fe2e828d4f939c1ffb329b70c239, user_id=5ecf899820154194a9ec0e9aeb6b56d4, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:22.476487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af345a52-1363-4f18-b181-60243bd8ac89 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.492343 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1308/2611] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:22 2022] PUT /identity/v3/projects/f122fe2e828d4f939c1ffb329b70c239/users/5ecf899820154194a9ec0e9aeb6b56d4/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:22.506896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.507430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.507927 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.555144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] MFA Rules not processed for user `5ecf899820154194a9ec0e9aeb6b56d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:22.556758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:22.594820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0d9e42b2284c4db2114b96a41aa067', 'request_id': 'req-e548fd9e-f5d0-467f-b776-113bf1f271e7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:22.601984 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0d9e42b2284c4db2114b96a41aa067', 'request_id': 'req-e548fd9e-f5d0-467f-b776-113bf1f271e7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:22.701244 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1309/2612] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:22.710580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.718527 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:22.750812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:22.753468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0d9e42b2284c4db2114b96a41aa067', 'request_id': 'req-e548fd9e-f5d0-467f-b776-113bf1f271e7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:22.769397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] The token cache is being invalidate because project 6d0d9e42b2284c4db2114b96a41aa067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:22.777537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:22.777537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:22.777537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1304/2613] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:22 2022] DELETE /identity/v3/projects/6d0d9e42b2284c4db2114b96a41aa067 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:22.777537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.777537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:22.809350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e7e11-e155-4d23-b99f-df42f6e338f2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:22.835225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:22.884517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:22.884517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.884517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.884517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.885646 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:22.888805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548fd9e-f5d0-467f-b776-113bf1f271e7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:22.895087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:22.896013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:22.896404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:22.896837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:22.906745 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:22.984306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:22.985490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:22.985983 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.988177 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:22.996872 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1305/2614] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:22.997799 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:22.999249 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:23.007775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1310/2615] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:23.235281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065adbd-f837-4f01-9fb4-9493ec6c8b08 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:23.241475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:23.242188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:23.242466 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:23.242793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] PATH_INFO: `/v3/users/0f61d8a4640b4fc68c0eff9e148d3eb4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:23.244509 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f61d8a4640b4fc68c0eff9e148d3eb4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:23.246639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:23.838025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f61d8a4640b4fc68c0eff9e148d3eb4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:23.838025 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f61d8a4640b4fc68c0eff9e148d3eb4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:23.853378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f61d8a4640b4fc68c0eff9e148d3eb4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:23.855078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f61d8a4640b4fc68c0eff9e148d3eb4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:23.948087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f61d8a4640b4fc68c0eff9e148d3eb4'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:23.964990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:23.977729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1306/2616] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:22 2022] DELETE /identity/v3/users/0f61d8a4640b4fc68c0eff9e148d3eb4 => generated 0 bytes in 980 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:24.646947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:24.663474 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2d967-7c12-40af-a9f3-0ec5ae137f62 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:24.744242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:24.746380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:24.778369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:24.780601 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:24.781243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:24.781682 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] PATH_INFO: `/v3/projects/2878504308be4564bddac45d36e8f198` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:24.783674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2878504308be4564bddac45d36e8f198)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:24.787264 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:24.837738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09de34-fb11-40ee-a769-e426d67451a4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:24.846232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:24.846865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:24.847332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:24.847806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] PATH_INFO: `/v3/users/1035be8bc7a949e2a49f5a77f5ac61cf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:24.849967 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1035be8bc7a949e2a49f5a77f5ac61cf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:24.853795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:24.929975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:24.949845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2878504308be4564bddac45d36e8f198', 'request_id': 'req-1449f199-8fee-4c73-9b05-52b9e97f2cd9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:24.958305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2878504308be4564bddac45d36e8f198', 'request_id': 'req-1449f199-8fee-4c73-9b05-52b9e97f2cd9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:24.965311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2878504308be4564bddac45d36e8f198', 'request_id': 'req-1449f199-8fee-4c73-9b05-52b9e97f2cd9'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.083123 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] The token cache is being invalidate because project 2878504308be4564bddac45d36e8f198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:25.083822 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.084947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1449f199-8fee-4c73-9b05-52b9e97f2cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.087206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1311/2617] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:24 2022] DELETE /identity/v3/projects/2878504308be4564bddac45d36e8f198 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:25.097878 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea21830-1764-453e-81c4-2b201e4f6b67 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.098786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea21830-1764-453e-81c4-2b201e4f6b67 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.099382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea21830-1764-453e-81c4-2b201e4f6b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.109741 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1035be8bc7a949e2a49f5a77f5ac61cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.110656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1035be8bc7a949e2a49f5a77f5ac61cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.122357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1035be8bc7a949e2a49f5a77f5ac61cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.124261 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1035be8bc7a949e2a49f5a77f5ac61cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.137070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-2ea21830-1764-453e-81c4-2b201e4f6b67 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:25.143378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-2ea21830-1764-453e-81c4-2b201e4f6b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:25.230349 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1312/2618] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:25.234343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-340f6b36-53b3-4657-9f7c-458c5a8f2f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1035be8bc7a949e2a49f5a77f5ac61cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:25.239387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527d906-68fe-49fd-81fc-a7a932eebb94 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.239861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527d906-68fe-49fd-81fc-a7a932eebb94 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.240278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527d906-68fe-49fd-81fc-a7a932eebb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.259362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1307/2619] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:24 2022] DELETE /identity/v3/users/1035be8bc7a949e2a49f5a77f5ac61cf => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:25.265253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.265514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.265788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.268019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d527d906-68fe-49fd-81fc-a7a932eebb94 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:25.270552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d527d906-68fe-49fd-81fc-a7a932eebb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:25.286565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:25.287780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:25.368724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1313/2620] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:25.371009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1308/2621] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:25.376141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527d906-68fe-49fd-81fc-a7a932eebb94 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:25.392781 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:25.393876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527d906-68fe-49fd-81fc-a7a932eebb94 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:25.399832 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:25.403888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:25.405917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.406447 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.407031 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.408215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:25.408736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:25.411775 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:25.422111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:25.439330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1314/2622] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:43:25.445575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:25.478047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a854d9-92a2-4c76-ab2c-4c39cdf7b612 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:25.478344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:25.479172 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.479624 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.480078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.481149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:25.485350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:25.502890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce3ef7-ae02-41bb-931d-33e90e4ab4e3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:25.518701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:25.518701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.518701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.518701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.531132 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:25.533595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:25.550579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1309/2623] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:25.613655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:25.615317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:25.687379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1315/2624] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:25 2022] POST /identity/v3/projects => generated 359 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:25.694508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:25.726383 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af8681-ef0a-4758-8874-30b3493b7712 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:25.742167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:25.797986 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064270fc-3c48-4c38-b909-d098a33883d6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:25.801500 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:25.802475 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.802888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.803253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.805313 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:25.806028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:25.807240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:25.808040 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:25.808528 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:25.809696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:25.814111 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:25.912544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:25.915521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:25.935563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1310/2625] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:25.973717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1316/2626] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:25 2022] POST /identity/v3/users => generated 280 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:25.985736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:26.010373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab22c10-c3c4-4cc1-af4e-b1ba1d80bcbe service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:26.016264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:26.017059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.017341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.017621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.018483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:26.020268 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:26.034096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1311/2627] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:26.038440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:26.050786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa79692-70c8-4256-b319-a274cd8421fe admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:26.055842 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:26.056494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.056790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.057222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.057901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:26.059625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:26.077382 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1317/2628] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:26.081958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:26.096742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c93a2-964f-4258-a854-81290941d555 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:26.102537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ab949-c5a8-4c06-bae1-9076d421ed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:26.103620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ab949-c5a8-4c06-bae1-9076d421ed46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.104086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ab949-c5a8-4c06-bae1-9076d421ed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.104476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ab949-c5a8-4c06-bae1-9076d421ed46 admin admin] PATH_INFO: `/v3/projects/8676a5e194fd456d98aae4d35da17c64/users/829becc3c9c44c899ecd84486f31ab8a/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.117236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:26.126646 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ab949-c5a8-4c06-bae1-9076d421ed46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8676a5e194fd456d98aae4d35da17c64, user_id=829becc3c9c44c899ecd84486f31ab8a, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:26.129595 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ab949-c5a8-4c06-bae1-9076d421ed46 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:26.269323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:26.269729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1312/2629] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:26 2022] PUT /identity/v3/projects/8676a5e194fd456d98aae4d35da17c64/users/829becc3c9c44c899ecd84486f31ab8a/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:26.269888 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:26.302384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf213e0-8f38-4e7f-81b2-d2add1cceb05 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.304550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b497ff-1ba0-4f14-b3e4-558bf245ed01 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:26.307026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf213e0-8f38-4e7f-81b2-d2add1cceb05 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.307969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf213e0-8f38-4e7f-81b2-d2add1cceb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.360764 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fbf213e0-8f38-4e7f-81b2-d2add1cceb05 None None] MFA Rules not processed for user `829becc3c9c44c899ecd84486f31ab8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:26.363229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fbf213e0-8f38-4e7f-81b2-d2add1cceb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:26.400734 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:26.401597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.402090 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.402512 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] PATH_INFO: `/v3/projects/7b7d4eb5c5d8433fa69a554c7abba6ab` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.406441 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7d4eb5c5d8433fa69a554c7abba6ab)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:26.408965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:26.472301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1313/2630] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:26.528824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf213e0-8f38-4e7f-81b2-d2add1cceb05 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:26.593555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf213e0-8f38-4e7f-81b2-d2add1cceb05 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:26.597977 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:26.598633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.598911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.599188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.612209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:26.613659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:26.624845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1314/2631] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:26.647691 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:26.660744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7d4eb5c5d8433fa69a554c7abba6ab', 'request_id': 'req-40e7be60-2d3b-407b-a33f-287ad6a47cb1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:26.666085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47634d50-c2b8-4fa9-a2ae-4b465179381e service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:26.672691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7d4eb5c5d8433fa69a554c7abba6ab', 'request_id': 'req-40e7be60-2d3b-407b-a33f-287ad6a47cb1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:26.686507 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7d4eb5c5d8433fa69a554c7abba6ab', 'request_id': 'req-40e7be60-2d3b-407b-a33f-287ad6a47cb1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:26.745343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:26.746323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.746841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.747205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] PATH_INFO: `/v3/users/829becc3c9c44c899ecd84486f31ab8a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.748830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=829becc3c9c44c899ecd84486f31ab8a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:26.750805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:26.778608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:26.838488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] The token cache is being invalidate because project 7b7d4eb5c5d8433fa69a554c7abba6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:26.839157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:26.839452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:26.841008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1318/2632] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:26 2022] DELETE /identity/v3/projects/7b7d4eb5c5d8433fa69a554c7abba6ab => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:26.846481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:26.848022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:26.861893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7be60-2d3b-407b-a33f-287ad6a47cb1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:26.867910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:26.868707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:26.869082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:26.869424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] PATH_INFO: `/v3/users/5ecf899820154194a9ec0e9aeb6b56d4` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:26.870835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ecf899820154194a9ec0e9aeb6b56d4)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:26.873757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:27.075013 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:27.126927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829becc3c9c44c899ecd84486f31ab8a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.128018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829becc3c9c44c899ecd84486f31ab8a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.135632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829becc3c9c44c899ecd84486f31ab8a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.136447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829becc3c9c44c899ecd84486f31ab8a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.216404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5f057a22-893f-4279-a1fd-ede2c03ecbbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829becc3c9c44c899ecd84486f31ab8a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.238506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ecf899820154194a9ec0e9aeb6b56d4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.239206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ecf899820154194a9ec0e9aeb6b56d4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.239831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ecf899820154194a9ec0e9aeb6b56d4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.242259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1315/2633] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:26 2022] DELETE /identity/v3/users/829becc3c9c44c899ecd84486f31ab8a => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:27.247487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:27.248567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:27.248874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:27.271156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:27.272506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:27.345738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1316/2634] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:27.363342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:27.363582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ecf899820154194a9ec0e9aeb6b56d4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.363743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:27.386182 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ecf899820154194a9ec0e9aeb6b56d4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:27.423606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1319/2635] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:26 2022] DELETE /identity/v3/users/5ecf899820154194a9ec0e9aeb6b56d4 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:27.473939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee061fb-4e34-4a45-927b-9fd22ec8c82c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:27.551520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:27.553126 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:27.553412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:27.553675 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:27.566537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:27.632770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:27.634501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:27.646412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1317/2636] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:28.120563 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:28.124908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:28.168487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c48c54-bee5-4fe5-8de7-d8e8ddb30ea6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:28.178346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:28.179399 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:28.180046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:28.180520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] PATH_INFO: `/v3/projects/f122fe2e828d4f939c1ffb329b70c239` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:28.187149 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f122fe2e828d4f939c1ffb329b70c239)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:28.189009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:28.405983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f122fe2e828d4f939c1ffb329b70c239', 'request_id': 'req-cacf80e9-04c2-44e7-a3ab-5e4869e89689'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.413298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f122fe2e828d4f939c1ffb329b70c239', 'request_id': 'req-cacf80e9-04c2-44e7-a3ab-5e4869e89689'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.422597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f122fe2e828d4f939c1ffb329b70c239', 'request_id': 'req-cacf80e9-04c2-44e7-a3ab-5e4869e89689'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.433009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:28.434888 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:28.452818 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce9d57-abdd-418d-a806-da305d5f032a service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:28.531260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:28.531920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:28.532205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:28.532473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] PATH_INFO: `/v3/projects/8676a5e194fd456d98aae4d35da17c64` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:28.533674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] RBAC: Authorizing `identity:delete_project(project_id=8676a5e194fd456d98aae4d35da17c64)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:28.535894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:28.546361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] The token cache is being invalidate because project f122fe2e828d4f939c1ffb329b70c239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:28.546797 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.548119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-cacf80e9-04c2-44e7-a3ab-5e4869e89689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.551536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1320/2637] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:28 2022] DELETE /identity/v3/projects/f122fe2e828d4f939c1ffb329b70c239 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:28.560851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a80efb-0a23-495b-9dcb-f99fe7a344be None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:28.561313 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a80efb-0a23-495b-9dcb-f99fe7a344be None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:28.561792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a80efb-0a23-495b-9dcb-f99fe7a344be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:28.591133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-75a80efb-0a23-495b-9dcb-f99fe7a344be None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:28.592942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-75a80efb-0a23-495b-9dcb-f99fe7a344be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:28.601724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1321/2638] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:28.610703 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b9072-7b1e-47ea-954b-acd302d8a196 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:28.610988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b9072-7b1e-47ea-954b-acd302d8a196 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:28.611262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b9072-7b1e-47ea-954b-acd302d8a196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:28.635131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-b74b9072-7b1e-47ea-954b-acd302d8a196 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:28.636444 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b74b9072-7b1e-47ea-954b-acd302d8a196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:28.646960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1322/2639] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:28.652006 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b9072-7b1e-47ea-954b-acd302d8a196 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:28.663055 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b9072-7b1e-47ea-954b-acd302d8a196 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:28.668083 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:28.668718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:28.668999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:28.669279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:28.670243 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:28.670800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:28.672516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:28.690610 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1323/2640] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:43:28.702215 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:28.712787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8676a5e194fd456d98aae4d35da17c64', 'request_id': 'req-61933cf3-f0ec-448b-b459-9d8fbc58d8db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.718471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e33595-6a36-4c45-b8b3-6ea6ca9f7dd4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:28.721183 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8676a5e194fd456d98aae4d35da17c64', 'request_id': 'req-61933cf3-f0ec-448b-b459-9d8fbc58d8db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.795145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:28.796117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:28.796442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:28.796784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:28.797568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:28.799714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:28.804794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:28.860504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8676a5e194fd456d98aae4d35da17c64', 'request_id': 'req-61933cf3-f0ec-448b-b459-9d8fbc58d8db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.873917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] The token cache is being invalidate because project 8676a5e194fd456d98aae4d35da17c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:28.874390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.875235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-61933cf3-f0ec-448b-b459-9d8fbc58d8db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:28.876845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1318/2641] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:28 2022] DELETE /identity/v3/projects/8676a5e194fd456d98aae4d35da17c64 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:28.882513 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6033c-fe16-4f26-a269-67645961c08d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:28.882801 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6033c-fe16-4f26-a269-67645961c08d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:28.883148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6033c-fe16-4f26-a269-67645961c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:28.909831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-98c6033c-fe16-4f26-a269-67645961c08d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:28.910141 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1324/2642] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:28 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:28.914209 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-98c6033c-fe16-4f26-a269-67645961c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:28.915811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:28.918361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:28.957715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:29.002694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1319/2643] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:29.015738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e1e23-043a-4fa9-bcaa-5e5f6c26dd5e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.016152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947a5e7-6f86-46f0-a82c-97b4d668d52e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.016686 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947a5e7-6f86-46f0-a82c-97b4d668d52e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.017148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947a5e7-6f86-46f0-a82c-97b4d668d52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.022688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.023354 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.023631 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.023903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.024584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.026632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.048439 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5947a5e7-6f86-46f0-a82c-97b4d668d52e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:29.050483 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5947a5e7-6f86-46f0-a82c-97b4d668d52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:29.064626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1320/2644] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:29.069523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947a5e7-6f86-46f0-a82c-97b4d668d52e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.081066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947a5e7-6f86-46f0-a82c-97b4d668d52e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.086752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.087453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.087851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.088234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.088986 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:29.089369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.091400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.106214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1321/2645] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:43:29.115477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.128373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c918cd-cbf2-48d9-b8ad-20db2996134e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.134141 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.135179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.135660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.136136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.137272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.140225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1325/2646] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:28 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:29.142773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.144191 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.158401 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ba13c-3375-4aea-8f7e-d509146b5c99 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.163702 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.164218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.164497 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.164773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.165452 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.167118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.181293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1326/2647] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:29.187058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.202361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2879cc-a735-490f-a278-bd7428410caa admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.209464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.210281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.210607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.210954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.211666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.213530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.229560 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1327/2648] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:29.236346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.252018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094378-fc43-4080-889d-c4a77cc2c9ec admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.259096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7a348-a5a4-4352-99aa-5755760654ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.259894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7a348-a5a4-4352-99aa-5755760654ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.260179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7a348-a5a4-4352-99aa-5755760654ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.260459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7a348-a5a4-4352-99aa-5755760654ce admin admin] PATH_INFO: `/v3/projects/c9fcbf7de2624f1f92be203cd978cd83/users/bb303dfcd6d34866bd7c875924ae86ae/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.285597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1322/2649] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:29 2022] POST /identity/v3/projects => generated 391 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:29.295309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7a348-a5a4-4352-99aa-5755760654ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fcbf7de2624f1f92be203cd978cd83, user_id=bb303dfcd6d34866bd7c875924ae86ae, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.297680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.298307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7a348-a5a4-4352-99aa-5755760654ce admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.425568 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8471-e8fb-4cb2-ae53-b862862c6eae admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.431017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.431654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.431935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.432205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.432831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.435768 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.500109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1328/2650] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:29 2022] PUT /identity/v3/projects/c9fcbf7de2624f1f92be203cd978cd83/users/bb303dfcd6d34866bd7c875924ae86ae/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:29.517245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0eea7-0fc2-455c-9c82-eb1830439804 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.517710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0eea7-0fc2-455c-9c82-eb1830439804 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.518238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0eea7-0fc2-455c-9c82-eb1830439804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.556312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-70c0eea7-0fc2-455c-9c82-eb1830439804 None None] MFA Rules not processed for user `bb303dfcd6d34866bd7c875924ae86ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:29.557592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-70c0eea7-0fc2-455c-9c82-eb1830439804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:29.577650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1323/2651] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:29 2022] POST /identity/v3/users => generated 296 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:29.582954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.603446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da531d1-b5c3-4a1e-919c-c3613947779f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.676832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1329/2652] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:29.681953 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0eea7-0fc2-455c-9c82-eb1830439804 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.722024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.730278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.731250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.733957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.735576 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.740149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.778117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1324/2653] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:29.780416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.807101 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8174df-2b4a-4b1a-a836-c3c1438e5e29 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.812747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.813647 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.814589 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.815582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.816433 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.818321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.833186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1325/2654] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:29.837848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:29.842904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0eea7-0fc2-455c-9c82-eb1830439804 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.850454 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe14d5-a835-4c15-9344-500e6c21793e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:29.851072 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce39baa-9882-409d-bc8f-492cf6c49d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.854875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce39baa-9882-409d-bc8f-492cf6c49d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.854875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce39baa-9882-409d-bc8f-492cf6c49d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.854875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce39baa-9882-409d-bc8f-492cf6c49d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.854875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce39baa-9882-409d-bc8f-492cf6c49d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.856250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:29.856956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:29.857238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:29.858162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] PATH_INFO: `/v3/projects/a6b5152073f748c287990d07c710f023/users/8909d92d6de34b26adc18a72742f1f29/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:29.858333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce39baa-9882-409d-bc8f-492cf6c49d50 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:29.876245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b5152073f748c287990d07c710f023, user_id=8909d92d6de34b26adc18a72742f1f29, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:29.879102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:30.242598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1326/2655] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:29 2022] PUT /identity/v3/projects/a6b5152073f748c287990d07c710f023/users/8909d92d6de34b26adc18a72742f1f29/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:30.247660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1330/2656] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:29 2022] POST /identity/v3/projects => generated 385 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:30.252158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:30.257478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.257765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.258145 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:30.297500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] MFA Rules not processed for user `8909d92d6de34b26adc18a72742f1f29`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:30.300853 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:30.352521 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4677acc-e7ba-4890-af49-d6fbe64346f4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:30.385171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1331/2657] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:30.399378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:30.401329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:30.441144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:30.442118 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.442582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.443053 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:30.444144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:30.447748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:30.492167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783896d-87d2-46c8-8f61-a6b12f2b72a2 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:30.496647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:30.497324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.498178 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.498450 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:30.509816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:30.515733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:30.522966 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1332/2658] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:30.551067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1327/2659] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:30 2022] POST /identity/v3/users => generated 299 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:30.552326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:30.564202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb336764-283b-4e3a-ba6f-e1e6bcb6d47f service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:30.569046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:30.569656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.570012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.570245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:30.570897 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:30.572559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:30.585674 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1333/2660] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:30.590761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:30.603416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb56e4f-cbd8-4557-8088-a9a047c5212c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:30.609798 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:30.610586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.610867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.611137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] PATH_INFO: `/v3/projects/774dfc638b6440738b7f23701b53a7f4/users/e507c3b493f245f8b9f0aa32e326c740/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:30.636140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=774dfc638b6440738b7f23701b53a7f4, user_id=e507c3b493f245f8b9f0aa32e326c740, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:30.641553 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:30.783864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1328/2661] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:30 2022] PUT /identity/v3/projects/774dfc638b6440738b7f23701b53a7f4/users/e507c3b493f245f8b9f0aa32e326c740/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:30.789334 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:30.801210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295cdbc-a06c-44a0-8728-5bfd18092ee3 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:30.882786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb8904-da38-4987-8a1d-e913051399d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:30.883995 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb8904-da38-4987-8a1d-e913051399d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.884463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb8904-da38-4987-8a1d-e913051399d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.885459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb8904-da38-4987-8a1d-e913051399d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:30.886567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb8904-da38-4987-8a1d-e913051399d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:30.891118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb8904-da38-4987-8a1d-e913051399d6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:30.918315 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1334/2662] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:30.918956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:30.936224 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5c859-4810-4ac1-b22c-deff5f044cab admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:30.944878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:30.946045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.946497 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.946894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] PATH_INFO: `/v3/projects/774dfc638b6440738b7f23701b53a7f4/users/e507c3b493f245f8b9f0aa32e326c740/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:30.950574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] RBAC: Authorizing `identity:create_grant(project_id=774dfc638b6440738b7f23701b53a7f4, user_id=e507c3b493f245f8b9f0aa32e326c740, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:30.957913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:30.979541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1329/2663] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:30 2022] PUT /identity/v3/projects/774dfc638b6440738b7f23701b53a7f4/users/e507c3b493f245f8b9f0aa32e326c740/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:30.998417 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbf890-7212-4ae2-8fe7-d7ca8152c39f None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:30.998988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbf890-7212-4ae2-8fe7-d7ca8152c39f None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:30.999583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbf890-7212-4ae2-8fe7-d7ca8152c39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:31.034156 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f0bbf890-7212-4ae2-8fe7-d7ca8152c39f None None] MFA Rules not processed for user `e507c3b493f245f8b9f0aa32e326c740`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:31.035658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f0bbf890-7212-4ae2-8fe7-d7ca8152c39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:31.122277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1335/2664] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:31.469377 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:31.538512 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577113-527d-4549-9a16-6656f25e899a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:31.542366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:31.543023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:31.543302 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:31.543571 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:31.555452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:31.556991 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:31.564109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:31.573770 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1330/2665] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:31.662887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:31.776430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbf890-7212-4ae2-8fe7-d7ca8152c39f None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:31.869693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:31.873575 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbf890-7212-4ae2-8fe7-d7ca8152c39f None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:31.875085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:31.883971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:31.885091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:31.885611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:31.886144 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] PATH_INFO: `/v3/users/bb303dfcd6d34866bd7c875924ae86ae` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:31.888081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb303dfcd6d34866bd7c875924ae86ae)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:31.892116 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:31.892374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770b736-74ad-4295-9e72-dfce47f242e2 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:31.898623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:31.899329 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:31.899613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:31.899885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] PATH_INFO: `/v3/users/18085b59d9614ca198de336505be931d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:31.901140 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] RBAC: Authorizing `identity:delete_user(user_id=18085b59d9614ca198de336505be931d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:31.903267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:32.211312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb303dfcd6d34866bd7c875924ae86ae'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.211964 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb303dfcd6d34866bd7c875924ae86ae'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.212594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb303dfcd6d34866bd7c875924ae86ae'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.263996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18085b59d9614ca198de336505be931d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.264823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18085b59d9614ca198de336505be931d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.272739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18085b59d9614ca198de336505be931d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.273538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18085b59d9614ca198de336505be931d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.373532 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb303dfcd6d34866bd7c875924ae86ae'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.385777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ee674b80-4686-4a0a-b955-3eaabffed421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb303dfcd6d34866bd7c875924ae86ae'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.407176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1336/2666] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:31 2022] DELETE /identity/v3/users/bb303dfcd6d34866bd7c875924ae86ae => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:32.411747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:32.412049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:32.412330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:32.436738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:32.438020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:32.514916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1337/2667] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:32.526314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:32.527746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:32.541677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f252c2f2-e5dc-4106-9bfc-e8227496e79a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18085b59d9614ca198de336505be931d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:32.571362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1331/2668] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:31 2022] DELETE /identity/v3/users/18085b59d9614ca198de336505be931d => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:32.578621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734777f-70cb-4509-8bf3-b4525bd32284 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:32.579166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734777f-70cb-4509-8bf3-b4525bd32284 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:32.579543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734777f-70cb-4509-8bf3-b4525bd32284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:32.604565 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9734777f-70cb-4509-8bf3-b4525bd32284 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:32.606215 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9734777f-70cb-4509-8bf3-b4525bd32284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:32.619669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7459d-424f-43a1-b12d-b2558be9fec1 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:32.696926 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:32.697736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:32.698130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:32.698483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:32.722712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1332/2669] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:32.735704 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734777f-70cb-4509-8bf3-b4525bd32284 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:32.752360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734777f-70cb-4509-8bf3-b4525bd32284 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:32.758952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:32.759931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:32.760375 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:32.760814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:32.776823 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:32.778641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:32.790750 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1333/2670] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:32.811565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:32.814270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:32.835546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1338/2671] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:32.894736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:32.995208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:33.485213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:33.488417 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:33.498980 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:33.501052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:33.526936 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc02f-1306-400f-87c6-c8c7a118bcef service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:33.529140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f640a7-f03e-40e5-a357-5d504d66865f service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:33.536533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:33.538868 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b6477-552f-407a-8769-090ee554347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:33.539895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b6477-552f-407a-8769-090ee554347d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:33.540076 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:33.540522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b6477-552f-407a-8769-090ee554347d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:33.540836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:33.541218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b6477-552f-407a-8769-090ee554347d admin admin] PATH_INFO: `/v3/projects/547369ecc41645b4822b0ddbd4cd3bd2` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:33.541412 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] PATH_INFO: `/v3/projects/c9fcbf7de2624f1f92be203cd978cd83` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:33.543566 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b6477-552f-407a-8769-090ee554347d admin admin] RBAC: Authorizing `identity:delete_project(project_id=547369ecc41645b4822b0ddbd4cd3bd2)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:33.546308 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fcbf7de2624f1f92be203cd978cd83)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:33.550343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:33.551213 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b6477-552f-407a-8769-090ee554347d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:33.726835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fcbf7de2624f1f92be203cd978cd83', 'request_id': 'req-0251222e-801b-4297-9568-3b6b0ca8e5ba'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.729829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547369ecc41645b4822b0ddbd4cd3bd2', 'request_id': 'req-737b6477-552f-407a-8769-090ee554347d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.737495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fcbf7de2624f1f92be203cd978cd83', 'request_id': 'req-0251222e-801b-4297-9568-3b6b0ca8e5ba'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.742643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547369ecc41645b4822b0ddbd4cd3bd2', 'request_id': 'req-737b6477-552f-407a-8769-090ee554347d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.750766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547369ecc41645b4822b0ddbd4cd3bd2', 'request_id': 'req-737b6477-552f-407a-8769-090ee554347d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.817089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fcbf7de2624f1f92be203cd978cd83', 'request_id': 'req-0251222e-801b-4297-9568-3b6b0ca8e5ba'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.836945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] The token cache is being invalidate because project c9fcbf7de2624f1f92be203cd978cd83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:33.837317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.838110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.839901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1334/2672] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:33 2022] DELETE /identity/v3/projects/c9fcbf7de2624f1f92be203cd978cd83 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:33.844852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:33.847090 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:33.939971 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251222e-801b-4297-9568-3b6b0ca8e5ba admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:33.974459 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-737b6477-552f-407a-8769-090ee554347d admin admin] The token cache is being invalidate because project 547369ecc41645b4822b0ddbd4cd3bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:33.975764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.977119 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:33.983229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1339/2673] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:33 2022] DELETE /identity/v3/projects/547369ecc41645b4822b0ddbd4cd3bd2 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:33.999590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:34.002844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:34.054649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:34.055572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:34.055951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:34.056322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] PATH_INFO: `/v3/users/e507c3b493f245f8b9f0aa32e326c740` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:34.057793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e507c3b493f245f8b9f0aa32e326c740)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:34.059914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:34.076282 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:34.123904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b6477-552f-407a-8769-090ee554347d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:34.129303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:34.130270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:34.130554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:34.130829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] PATH_INFO: `/v3/users/1f3895eb010e41daac2d9f80456d8810` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:34.132065 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3895eb010e41daac2d9f80456d8810)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:34.134337 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:34.222789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:34.396660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e507c3b493f245f8b9f0aa32e326c740'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.397889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e507c3b493f245f8b9f0aa32e326c740'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.412028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3895eb010e41daac2d9f80456d8810'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.412800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3895eb010e41daac2d9f80456d8810'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.413204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e507c3b493f245f8b9f0aa32e326c740'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.413681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3895eb010e41daac2d9f80456d8810'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.414338 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e507c3b493f245f8b9f0aa32e326c740'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.537391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3895eb010e41daac2d9f80456d8810'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.547020 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3895eb010e41daac2d9f80456d8810'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.571564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1340/2674] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:33 2022] DELETE /identity/v3/users/1f3895eb010e41daac2d9f80456d8810 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:34.604911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e507c3b493f245f8b9f0aa32e326c740'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:34.631355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1335/2675] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:33 2022] DELETE /identity/v3/users/e507c3b493f245f8b9f0aa32e326c740 => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:34.648268 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:35.273438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:35.275636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:35.297204 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:35.311243 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:35.317326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:35.405483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd90832-11a5-4259-be14-db8a7b7789e7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:35.417859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6648e41-9faa-4d8d-9116-fc45f9b0f423 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:35.423258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:35.423950 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:35.424228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:35.424505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] PATH_INFO: `/v3/projects/774dfc638b6440738b7f23701b53a7f4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:35.425942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] RBAC: Authorizing `identity:delete_project(project_id=774dfc638b6440738b7f23701b53a7f4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:35.427903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:35.500777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:35.501568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:35.501948 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:35.502494 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] PATH_INFO: `/v3/projects/129b7e8fc5244ec3b7078e80e745f3a9` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:35.504239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129b7e8fc5244ec3b7078e80e745f3a9)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:35.507389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:35.576446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774dfc638b6440738b7f23701b53a7f4', 'request_id': 'req-f2dd458b-b635-41a0-8708-9af2780f331c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.583919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774dfc638b6440738b7f23701b53a7f4', 'request_id': 'req-f2dd458b-b635-41a0-8708-9af2780f331c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.688660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129b7e8fc5244ec3b7078e80e745f3a9', 'request_id': 'req-f810748d-7141-4b77-94ba-024de84dab90'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.694790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129b7e8fc5244ec3b7078e80e745f3a9', 'request_id': 'req-f810748d-7141-4b77-94ba-024de84dab90'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.705037 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129b7e8fc5244ec3b7078e80e745f3a9', 'request_id': 'req-f810748d-7141-4b77-94ba-024de84dab90'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.736883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774dfc638b6440738b7f23701b53a7f4', 'request_id': 'req-f2dd458b-b635-41a0-8708-9af2780f331c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.748155 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:35.758842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] The token cache is being invalidate because project 774dfc638b6440738b7f23701b53a7f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:35.759364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.760533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f2dd458b-b635-41a0-8708-9af2780f331c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:35.763637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1336/2676] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:35 2022] DELETE /identity/v3/projects/774dfc638b6440738b7f23701b53a7f4 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:35.770867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce99a0-0164-4e20-8768-3dd140545aee None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:35.771277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce99a0-0164-4e20-8768-3dd140545aee None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:35.771670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce99a0-0164-4e20-8768-3dd140545aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:35.801047 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8cce99a0-0164-4e20-8768-3dd140545aee None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:35.802962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8cce99a0-0164-4e20-8768-3dd140545aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:35.880918 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1337/2677] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:35.889119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3776-8410-4cc8-b567-dffdd58aa335 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:35.889598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3776-8410-4cc8-b567-dffdd58aa335 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:35.889869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3776-8410-4cc8-b567-dffdd58aa335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:35.912574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e64c3776-8410-4cc8-b567-dffdd58aa335 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:35.914104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e64c3776-8410-4cc8-b567-dffdd58aa335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:35.924763 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1338/2678] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:35.929092 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c3776-8410-4cc8-b567-dffdd58aa335 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:35.940362 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c3776-8410-4cc8-b567-dffdd58aa335 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:35.946149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:35.946852 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:35.947166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:35.947494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:35.948236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:35.948577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:35.951073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:35.965514 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1339/2679] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:43:35.971058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:35.983212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca010915-88c6-4c92-896a-b8ce996e12a5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:35.989358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:35.990156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:35.990495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:35.990921 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:35.991700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:35.994094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:36.206235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] The token cache is being invalidate because project 129b7e8fc5244ec3b7078e80e745f3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:36.206639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:36.207414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:36.207879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1340/2680] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:35 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:36.212662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1341/2681] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:35 2022] DELETE /identity/v3/projects/129b7e8fc5244ec3b7078e80e745f3a9 => generated 0 bytes in 940 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:36.213671 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:36.215634 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:36.217788 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:36.219688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:36.237054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810748d-7141-4b77-94ba-024de84dab90 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:36.308644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:36.309458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:36.309715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:36.310578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] PATH_INFO: `/v3/users/eaad53e142474c31824fbe766b349fda` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:36.312737 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaad53e142474c31824fbe766b349fda)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:36.314284 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f37fe-7334-46fc-8924-a949ade21c13 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:36.314784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:36.322486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:36.323664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:36.324156 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:36.324616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:36.325809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:36.330626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:36.491930 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1341/2682] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:36 2022] POST /identity/v3/users => generated 290 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:36.496557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:36.511151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77c8d7-c9e9-474c-99e6-c2f4bd89d0ca admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:36.519225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:36.520316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:36.520828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:36.521334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:36.522446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:36.525267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:36.546797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1342/2683] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:36.554309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:36.565451 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6471c2d-0492-4188-bfaa-24cb82987bb3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:36.570681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:36.571395 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:36.571715 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:36.572027 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:36.572724 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:36.574651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:36.587690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1343/2684] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:36.595045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:36.605842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f531d3-6343-4560-8c2d-37ce4ba8fe0c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:36.611867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47199658-c46c-4364-b425-fc7deb140c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:36.612566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47199658-c46c-4364-b425-fc7deb140c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:36.612846 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47199658-c46c-4364-b425-fc7deb140c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:36.613117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47199658-c46c-4364-b425-fc7deb140c78 admin admin] PATH_INFO: `/v3/projects/d511a51be1fe4493a8f3b7c035cb373a/users/d085734268f14bac825be1e7cf7d27dd/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:36.630855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47199658-c46c-4364-b425-fc7deb140c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d511a51be1fe4493a8f3b7c035cb373a, user_id=d085734268f14bac825be1e7cf7d27dd, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:36.633736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47199658-c46c-4364-b425-fc7deb140c78 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:36.680386 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaad53e142474c31824fbe766b349fda'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:36.681413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaad53e142474c31824fbe766b349fda'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:36.682370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaad53e142474c31824fbe766b349fda'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:36.690631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:36.749505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1344/2685] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:36 2022] PUT /identity/v3/projects/d511a51be1fe4493a8f3b7c035cb373a/users/d085734268f14bac825be1e7cf7d27dd/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:36.763981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e50e0-744f-49cc-a5bb-4b4c454eab40 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:36.765381 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaad53e142474c31824fbe766b349fda'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:36.770096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e50e0-744f-49cc-a5bb-4b4c454eab40 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:36.770096 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e50e0-744f-49cc-a5bb-4b4c454eab40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:36.778435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaad53e142474c31824fbe766b349fda'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:36.804713 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cc1e50e0-744f-49cc-a5bb-4b4c454eab40 None None] MFA Rules not processed for user `d085734268f14bac825be1e7cf7d27dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:36.805355 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1342/2686] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:36 2022] DELETE /identity/v3/users/eaad53e142474c31824fbe766b349fda => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:36.806456 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cc1e50e0-744f-49cc-a5bb-4b4c454eab40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:36.881046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:36.914965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1345/2687] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:36.920175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:36.922295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:37.003832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829037-c003-4598-88bf-8e678e617f90 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:37.008864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:37.009488 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.009770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.010098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.010725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:37.012667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:37.137783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1343/2688] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:36 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:37.143642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e50e0-744f-49cc-a5bb-4b4c454eab40 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:37.160859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e50e0-744f-49cc-a5bb-4b4c454eab40 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:37.179962 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:37.235877 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:37.238083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.238083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.238083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.238530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:37.240189 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:37.350828 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1346/2689] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:37 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:37.355360 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:37.375496 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81205c-407d-4c04-af34-1c2283c9db9e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:37.383536 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:37.384475 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.384923 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.385256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.386192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:37.389968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:37.404504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1344/2690] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:37.411660 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:37.423447 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065ebc3-185a-48b2-a9f7-4553c72caf2b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:37.428600 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:37.429222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.429517 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.429765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.430460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:37.432125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:37.447836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1347/2691] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:37.453418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:37.460226 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:37.465026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cec9a-b5be-4732-9662-c7723d24323d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:37.470608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffaa53-a6cb-49a6-927a-ad36f3003de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:37.471284 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffaa53-a6cb-49a6-927a-ad36f3003de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.471562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffaa53-a6cb-49a6-927a-ad36f3003de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.471835 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffaa53-a6cb-49a6-927a-ad36f3003de7 admin admin] PATH_INFO: `/v3/projects/d48bb958f0774173868d792de7b7a791/users/836f5511ea3940529a61b16d2b9f12db/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.490651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffaa53-a6cb-49a6-927a-ad36f3003de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48bb958f0774173868d792de7b7a791, user_id=836f5511ea3940529a61b16d2b9f12db, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:37.493730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffaa53-a6cb-49a6-927a-ad36f3003de7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:37.496649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:37.500380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:37.524511 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f2bfe-33b0-423c-8f81-fb68aa7ac79b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:37.532181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:37.532980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.533316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.533587 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] PATH_INFO: `/v3/projects/6bec9455cd734c5da56798503cf66c35` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.534767 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bec9455cd734c5da56798503cf66c35)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:37.536722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:37.736670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1345/2692] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:37 2022] PUT /identity/v3/projects/d48bb958f0774173868d792de7b7a791/users/836f5511ea3940529a61b16d2b9f12db/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:37.752963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b205-b775-40a2-aebf-f55416edb08d None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.753456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b205-b775-40a2-aebf-f55416edb08d None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.754021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b205-b775-40a2-aebf-f55416edb08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.795582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bec9455cd734c5da56798503cf66c35', 'request_id': 'req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:37.803695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1268b205-b775-40a2-aebf-f55416edb08d None None] MFA Rules not processed for user `836f5511ea3940529a61b16d2b9f12db`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:37.814043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1268b205-b775-40a2-aebf-f55416edb08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:37.815725 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bec9455cd734c5da56798503cf66c35', 'request_id': 'req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:37.892860 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1346/2693] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:37.897694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268b205-b775-40a2-aebf-f55416edb08d None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:37.964989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bec9455cd734c5da56798503cf66c35', 'request_id': 'req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:37.972319 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268b205-b775-40a2-aebf-f55416edb08d None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:37.977522 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:37.978197 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:37.978523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:37.978803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:37.979372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:37.981350 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:37.993633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] The token cache is being invalidate because project 6bec9455cd734c5da56798503cf66c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:37.994181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:37.995222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:37.999073 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1348/2694] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:37 2022] DELETE /identity/v3/projects/6bec9455cd734c5da56798503cf66c35 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:38.099199 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1347/2695] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:37 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:38.100573 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:38.102368 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:38.117540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4db87-9f9f-46d7-b1d6-1935f37c00a2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:38.197151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:38.198755 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:38.199033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:38.199535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:38.201346 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:38.205203 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:38.405737 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1349/2696] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:38 2022] POST /identity/v3/users => generated 295 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:38.412170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:38.424429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ef33b-cfdb-4764-9ef3-b5003893ff89 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:38.429484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:38.430227 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:38.430505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:38.430772 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:38.431416 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:38.433015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:38.444911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1348/2697] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:38.449585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:38.461217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83c882-77bd-4b35-8bc4-b4183b3a1a4b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:38.466095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:38.466785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:38.467066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:38.467342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] PATH_INFO: `/v3/projects/3473da8a61564f64a45ab615a8db425d/users/b8c3cef83055450491e18849739fdcd6/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:38.471114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:38.485002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3473da8a61564f64a45ab615a8db425d, user_id=b8c3cef83055450491e18849739fdcd6, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:38.489426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:38.636537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1350/2698] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:38 2022] PUT /identity/v3/projects/3473da8a61564f64a45ab615a8db425d/users/b8c3cef83055450491e18849739fdcd6/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:38.641609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:38.660363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6912664-9a55-4055-a490-6399153b1dd9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:38.719772 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:38.746523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d9d66-2c6d-4456-b5f4-2a034af673ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:38.747573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d9d66-2c6d-4456-b5f4-2a034af673ea admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:38.748867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d9d66-2c6d-4456-b5f4-2a034af673ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:38.749370 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d9d66-2c6d-4456-b5f4-2a034af673ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:38.750625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d9d66-2c6d-4456-b5f4-2a034af673ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:38.754049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d9d66-2c6d-4456-b5f4-2a034af673ea admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:38.776592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1349/2699] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:38.778147 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:38.789893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecc9b1-6491-4501-b758-bc51e2137c40 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:38.796543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:38.797295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:38.797615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:38.797946 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] PATH_INFO: `/v3/projects/3473da8a61564f64a45ab615a8db425d/users/b8c3cef83055450491e18849739fdcd6/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:38.800019 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3473da8a61564f64a45ab615a8db425d, user_id=b8c3cef83055450491e18849739fdcd6, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:38.802693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:38.814014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1351/2700] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:38 2022] PUT /identity/v3/projects/3473da8a61564f64a45ab615a8db425d/users/b8c3cef83055450491e18849739fdcd6/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:38.824634 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:38.824924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:38.825218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:38.853949 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] MFA Rules not processed for user `b8c3cef83055450491e18849739fdcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:38.855223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:38.933208 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1350/2701] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:38.945389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:38.947127 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:39.014803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb941277-3d7e-472c-b64f-56ff46bb4acb admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:39.018987 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:39.019803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:39.020098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:39.020343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:39.031620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:39.033068 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:39.043988 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1352/2702] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:39.206276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:39.208277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:39.222735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c3fdb-3aaa-4541-9927-59a8bfe2b5fe None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:39.231487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bb05b-de3a-416d-851e-c4164be12fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:39.232163 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb05b-de3a-416d-851e-c4164be12fcf service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:39.233188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb05b-de3a-416d-851e-c4164be12fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:39.233188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb05b-de3a-416d-851e-c4164be12fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:39.234730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-638bb05b-de3a-416d-851e-c4164be12fcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:39.305478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bb05b-de3a-416d-851e-c4164be12fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:39.307154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bb05b-de3a-416d-851e-c4164be12fcf service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:39.323195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1351/2703] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:48.391455 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:48.487474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4ce0-d67c-4549-bc55-73b61fd38e7f service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:48.494152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:48.494152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:48.494152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:48.494152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] PATH_INFO: `/v3/users/d085734268f14bac825be1e7cf7d27dd` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:48.495740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d085734268f14bac825be1e7cf7d27dd)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:48.498331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:48.757620 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd085734268f14bac825be1e7cf7d27dd'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:48.758370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd085734268f14bac825be1e7cf7d27dd'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:48.769021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd085734268f14bac825be1e7cf7d27dd'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:48.769812 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd085734268f14bac825be1e7cf7d27dd'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:48.861864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd085734268f14bac825be1e7cf7d27dd'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:48.878030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:48.896933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1353/2704] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:48 2022] DELETE /identity/v3/users/d085734268f14bac825be1e7cf7d27dd => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:48.900944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:48.901816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:48.901816 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:48.939706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:48.941809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:49.053479 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1352/2705] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:49.071736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:49.071736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:49.172010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f1bcc-a10e-4c50-9d6d-41dbbfee8cb6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:49.176923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:49.177955 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:49.178470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:49.178941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:49.195017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:49.197098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:49.210239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1354/2706] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:49.863457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:49.865708 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:49.882327 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=100 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:49.888805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d7f3-69c4-434d-8630-32b232071ddf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:49.895339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:49.897429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:49.897941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:49.898484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] PATH_INFO: `/v3/projects/d511a51be1fe4493a8f3b7c035cb373a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:49.900849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d511a51be1fe4493a8f3b7c035cb373a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:49.905046 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:50.042042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd511a51be1fe4493a8f3b7c035cb373a', 'request_id': 'req-dccd1a6f-df90-4131-9020-511b84afda03'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.050515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd511a51be1fe4493a8f3b7c035cb373a', 'request_id': 'req-dccd1a6f-df90-4131-9020-511b84afda03'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.061042 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd511a51be1fe4493a8f3b7c035cb373a', 'request_id': 'req-dccd1a6f-df90-4131-9020-511b84afda03'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.164397 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] The token cache is being invalidate because project d511a51be1fe4493a8f3b7c035cb373a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:50.164710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.165279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.166849 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1353/2707] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:49 2022] DELETE /identity/v3/projects/d511a51be1fe4493a8f3b7c035cb373a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:50.172793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:50.186337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c900780-a25a-4d66-a298-e16bca3de76e service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:50.256700 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:50.257522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:50.257902 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:50.258331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] PATH_INFO: `/v3/users/836f5511ea3940529a61b16d2b9f12db` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:50.259820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=836f5511ea3940529a61b16d2b9f12db)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:50.262086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:50.526094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836f5511ea3940529a61b16d2b9f12db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.526824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836f5511ea3940529a61b16d2b9f12db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.536012 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836f5511ea3940529a61b16d2b9f12db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.536838 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836f5511ea3940529a61b16d2b9f12db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.684478 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836f5511ea3940529a61b16d2b9f12db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:50.708161 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1355/2708] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:50 2022] DELETE /identity/v3/users/836f5511ea3940529a61b16d2b9f12db => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:51.293545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:51.295777 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:51.319009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd1a6f-df90-4131-9020-511b84afda03 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:51.402711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:51.405978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:51.418187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:51.418855 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:51.419192 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:51.419588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] PATH_INFO: `/v3/projects/d48bb958f0774173868d792de7b7a791` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:51.420938 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48bb958f0774173868d792de7b7a791)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:51.422988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:51.441892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:51.510002 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8d241-93a4-44c3-a1e1-2c1c816ae347 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:51.519631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:51.520355 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:51.520678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:51.521030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] PATH_INFO: `/v3/users/c1202f375e89427ba1c20340b5653aa1` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:51.522544 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1202f375e89427ba1c20340b5653aa1)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:51.524719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:51.564374 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48bb958f0774173868d792de7b7a791', 'request_id': 'req-f5ecc79a-b477-4204-9231-14edeefe0319'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.570553 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48bb958f0774173868d792de7b7a791', 'request_id': 'req-f5ecc79a-b477-4204-9231-14edeefe0319'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.579843 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48bb958f0774173868d792de7b7a791', 'request_id': 'req-f5ecc79a-b477-4204-9231-14edeefe0319'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.678871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] The token cache is being invalidate because project d48bb958f0774173868d792de7b7a791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:51.679196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.679893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.681480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1354/2709] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:51 2022] DELETE /identity/v3/projects/d48bb958f0774173868d792de7b7a791 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:51.685848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:51.687323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:51.710421 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:51.762396 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecc79a-b477-4204-9231-14edeefe0319 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:51.771198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:51.772346 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:51.772815 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:51.773317 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] PATH_INFO: `/v3/users/b8c3cef83055450491e18849739fdcd6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:51.775170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c3cef83055450491e18849739fdcd6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:51.777561 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:51.805786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1202f375e89427ba1c20340b5653aa1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.805786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1202f375e89427ba1c20340b5653aa1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.819452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1202f375e89427ba1c20340b5653aa1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.820538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1202f375e89427ba1c20340b5653aa1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.913476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8af5927-e2a5-495c-aded-c1121eb73161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1202f375e89427ba1c20340b5653aa1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:51.942652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1356/2710] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:51 2022] DELETE /identity/v3/users/c1202f375e89427ba1c20340b5653aa1 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:51.949050 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407589c0-2506-422e-b983-475f543f2279 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:51.949494 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407589c0-2506-422e-b983-475f543f2279 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:51.949961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407589c0-2506-422e-b983-475f543f2279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:51.975826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-407589c0-2506-422e-b983-475f543f2279 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:51.977208 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-407589c0-2506-422e-b983-475f543f2279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:52.049960 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c3cef83055450491e18849739fdcd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:52.050786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c3cef83055450491e18849739fdcd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:52.051537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c3cef83055450491e18849739fdcd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:52.055146 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1357/2711] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:52.067719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407589c0-2506-422e-b983-475f543f2279 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:52.069279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-407589c0-2506-422e-b983-475f543f2279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:52.117826 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:52.140143 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407589c0-2506-422e-b983-475f543f2279 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:52.144796 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:52.145654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:52.146102 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:52.150731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:52.150731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:52.164345 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:52.165913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:52.170845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c3cef83055450491e18849739fdcd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:52.178787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1358/2712] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:52.192568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c3cef83055450491e18849739fdcd6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:52.216668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1355/2713] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:51 2022] DELETE /identity/v3/users/b8c3cef83055450491e18849739fdcd6 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:52.772744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:52.775469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:52.806671 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:52.810983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:52.812322 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aacc5d-c1d6-4ff0-835b-9366b124ef13 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:52.929965 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:52.931191 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:52.931839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:52.932263 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] PATH_INFO: `/v3/projects/3473da8a61564f64a45ab615a8db425d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:52.933616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450782ac-0295-4cc6-a1d6-1dfdcd512c7f admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:52.933860 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3473da8a61564f64a45ab615a8db425d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:52.937326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:52.941607 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:52.942758 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:52.943214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:52.943654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] PATH_INFO: `/v3/projects/c2e84676253f422f9459952bfe7b006f` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:52.945482 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e84676253f422f9459952bfe7b006f)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:52.948082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.102787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3473da8a61564f64a45ab615a8db425d', 'request_id': 'req-7d179805-c1aa-44e8-9956-9e119306b1ba'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.110976 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3473da8a61564f64a45ab615a8db425d', 'request_id': 'req-7d179805-c1aa-44e8-9956-9e119306b1ba'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.138760 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:53.147881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e84676253f422f9459952bfe7b006f', 'request_id': 'req-458138c4-0eac-42dc-aa93-c3921f6c3e6f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.154570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e84676253f422f9459952bfe7b006f', 'request_id': 'req-458138c4-0eac-42dc-aa93-c3921f6c3e6f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.163008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e84676253f422f9459952bfe7b006f', 'request_id': 'req-458138c4-0eac-42dc-aa93-c3921f6c3e6f'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.238486 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3473da8a61564f64a45ab615a8db425d', 'request_id': 'req-7d179805-c1aa-44e8-9956-9e119306b1ba'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.253369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] The token cache is being invalidate because project 3473da8a61564f64a45ab615a8db425d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:53.253665 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.254627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7d179805-c1aa-44e8-9956-9e119306b1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.256300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1359/2714] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:52 2022] DELETE /identity/v3/projects/3473da8a61564f64a45ab615a8db425d => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:53.263890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370090-ea40-4ce6-8fa0-3a63052e5c75 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.264344 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370090-ea40-4ce6-8fa0-3a63052e5c75 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.264579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370090-ea40-4ce6-8fa0-3a63052e5c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.289059 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-12370090-ea40-4ce6-8fa0-3a63052e5c75 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:53.290684 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-12370090-ea40-4ce6-8fa0-3a63052e5c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:53.394735 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1360/2715] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:53.397625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] The token cache is being invalidate because project c2e84676253f422f9459952bfe7b006f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:53.398590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.403402 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-458138c4-0eac-42dc-aa93-c3921f6c3e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:53.405722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f950-3b70-4969-a4d7-8b58ce075427 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.406042 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f950-3b70-4969-a4d7-8b58ce075427 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.406556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f950-3b70-4969-a4d7-8b58ce075427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.407344 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1356/2716] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:52 2022] DELETE /identity/v3/projects/c2e84676253f422f9459952bfe7b006f => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:53.421662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65776d1-64f4-4536-979c-79498771560a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.422265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65776d1-64f4-4536-979c-79498771560a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.422700 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65776d1-64f4-4536-979c-79498771560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.436667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c509f950-3b70-4969-a4d7-8b58ce075427 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:53.439088 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c509f950-3b70-4969-a4d7-8b58ce075427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:53.452893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1361/2717] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:53.453595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f65776d1-64f4-4536-979c-79498771560a None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:53.455998 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f65776d1-64f4-4536-979c-79498771560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:53.457802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509f950-3b70-4969-a4d7-8b58ce075427 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.462800 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:53.469641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509f950-3b70-4969-a4d7-8b58ce075427 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.479210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1357/2718] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:53.479547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.479547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.479547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.479547 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.480187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:53.480187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.480187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.482294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb0c93-4e6d-4523-9d54-355c8dc2dc76 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.482294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb0c93-4e6d-4523-9d54-355c8dc2dc76 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.482979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb0c93-4e6d-4523-9d54-355c8dc2dc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.495030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1362/2719] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:43:53.501390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.516179 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211554d5-e20c-4b94-81a4-8b1559506368 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.518664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-83cb0c93-4e6d-4523-9d54-355c8dc2dc76 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:53.520771 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-83cb0c93-4e6d-4523-9d54-355c8dc2dc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:53.522323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.523324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.523797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.524259 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.525339 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.529135 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.535957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1358/2720] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:53.543290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb0c93-4e6d-4523-9d54-355c8dc2dc76 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.560578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb0c93-4e6d-4523-9d54-355c8dc2dc76 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.567725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.568517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.568846 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.569320 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.570293 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:53.570852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.572870 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.588077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1359/2721] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:43:53.596019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.607446 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac8291-7b50-47a4-b7b0-3c636129a3da admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.615606 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.616580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.617034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.617490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.618602 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.622985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.628399 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1363/2722] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:53.630937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.719138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c47357-1ead-4000-bb69-1689c55c04aa admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.725935 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.726591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.726864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.727128 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.727761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.730083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.776205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1360/2723] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:53.777723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.859063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1364/2724] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:53.859892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea7eeb-8214-42f4-83ae-c79adb55a7da admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.863281 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.865639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.866582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.867091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.867568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.868666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.872858 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.877958 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73f97b-0ad7-45ce-91f8-b7a0a4cc0de6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.885819 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.886225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.886508 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.886778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.887575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.889234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.901315 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1365/2725] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:53.905536 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.915867 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168020cb-1459-4226-9eca-289ec67c2bc7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.921758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.922622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.923066 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.923480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.924332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.926223 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:53.939585 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1366/2726] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:53.945851 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:53.957550 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4332c5f-c60a-4808-bb26-b740e6c552f0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:53.963348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b457bf-65be-4eec-a09c-0fc6d099929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:53.964109 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b457bf-65be-4eec-a09c-0fc6d099929f admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:53.964385 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b457bf-65be-4eec-a09c-0fc6d099929f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:53.964658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b457bf-65be-4eec-a09c-0fc6d099929f admin admin] PATH_INFO: `/v3/projects/5d15de97c1c54e06a44d85591005899f/users/f56a56d562b34e95b8864fc3bc0f6c57/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:53.993879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b457bf-65be-4eec-a09c-0fc6d099929f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d15de97c1c54e06a44d85591005899f, user_id=f56a56d562b34e95b8864fc3bc0f6c57, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:53.999726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1361/2727] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:53 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:54.005106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b457bf-65be-4eec-a09c-0fc6d099929f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:54.006249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:54.024670 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6def-02ea-4ffe-98bc-52804786e1f1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:54.032009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:54.032685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.033019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.033338 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.034176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:54.036314 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:54.056883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1362/2728] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:54.065885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:54.078985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c17b3-fd35-4e0e-ab55-167ff83a4035 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:54.087092 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:54.088298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.088800 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.089300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.090597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:54.093139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:54.113864 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1363/2729] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:54.114201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1367/2730] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:53 2022] PUT /identity/v3/projects/5d15de97c1c54e06a44d85591005899f/users/f56a56d562b34e95b8864fc3bc0f6c57/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:54.115261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:54.132440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b144b81-a88c-4516-9173-28b131ec1dc9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:54.132901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154a10e-8681-43e4-8d3b-ba661fd87fbd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.133086 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154a10e-8681-43e4-8d3b-ba661fd87fbd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.133257 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154a10e-8681-43e4-8d3b-ba661fd87fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.182232 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7154a10e-8681-43e4-8d3b-ba661fd87fbd None None] MFA Rules not processed for user `f56a56d562b34e95b8864fc3bc0f6c57`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:54.185113 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7154a10e-8681-43e4-8d3b-ba661fd87fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:54.198308 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:54.229269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c01cf-da29-406b-a30c-8b4a0926bb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:54.233232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c01cf-da29-406b-a30c-8b4a0926bb09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.233232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c01cf-da29-406b-a30c-8b4a0926bb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.233232 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c01cf-da29-406b-a30c-8b4a0926bb09 admin admin] PATH_INFO: `/v3/projects/5b24f59c5d7a45cbb034535eba660e7a/users/620a40c0e83449a6b39821a589add42c/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.261022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c01cf-da29-406b-a30c-8b4a0926bb09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b24f59c5d7a45cbb034535eba660e7a, user_id=620a40c0e83449a6b39821a589add42c, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:54.264307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c01cf-da29-406b-a30c-8b4a0926bb09 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:54.317960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:54.322228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1368/2731] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:54.328598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154a10e-8681-43e4-8d3b-ba661fd87fbd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:54.348556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154a10e-8681-43e4-8d3b-ba661fd87fbd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:54.349234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1364/2732] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:54 2022] PUT /identity/v3/projects/5b24f59c5d7a45cbb034535eba660e7a/users/620a40c0e83449a6b39821a589add42c/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:54.371388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.371388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.371388 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.408501 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] MFA Rules not processed for user `620a40c0e83449a6b39821a589add42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:54.409934 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:54.483898 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:54.484518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.484791 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.485061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.485842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:54.488989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:54.522573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1365/2733] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:43:54.530859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:54.530859 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:54.555643 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82f6de-54d2-41e8-83f0-6d5477510d3c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:54.561367 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:54.562216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.562572 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.562990 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] PATH_INFO: `/v3/users/8909d92d6de34b26adc18a72742f1f29` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.564689 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=8909d92d6de34b26adc18a72742f1f29)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:54.568107 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:54.600887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1369/2734] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:54 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:54.611858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:54.616327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:54.727309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba59260-19ab-4125-aaa1-fe957776f4d8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:54.732797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:54.733638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.734122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.734278 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:54.822209 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8909d92d6de34b26adc18a72742f1f29'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:54.823134 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8909d92d6de34b26adc18a72742f1f29'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:54.824167 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8909d92d6de34b26adc18a72742f1f29'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:54.824859 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:54.828307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:54.842880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1370/2735] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:54.847611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:54.916515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:54.922140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8909d92d6de34b26adc18a72742f1f29'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:54.938637 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0ef721bc-6da1-4ab5-8429-ff8a6f9c19ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8909d92d6de34b26adc18a72742f1f29'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:54.975523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1366/2736] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:54 2022] DELETE /identity/v3/users/8909d92d6de34b26adc18a72742f1f29 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:54.980766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2b47-b4a4-46b8-9d73-5d40a277858f service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:54.981084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:54.983787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:54.983787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.009049 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:55.011211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:55.063977 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:55.105352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:55.106836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.107871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.107871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.108892 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:55.112711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:55.130332 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1367/2737] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:55.141710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:55.143414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:55.222587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92392263-0cf2-4483-92e5-cdbe62365a13 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:55.230651 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:55.231839 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.232179 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.232697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.239830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1371/2738] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:54 2022] POST /identity/v3/users => generated 299 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:55.245431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:55.248739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:55.251969 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:55.253852 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:55.268468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dc1b1-8b5c-45d7-968e-a508f06e5219 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:55.269200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1368/2739] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:55.275992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:55.276886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.277216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.277596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.279152 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:55.281014 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:55.295334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1372/2740] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:55.300548 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:55.312691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ed037-46b3-4358-8f6e-aef8cbce62c2 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:55.320503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:55.322078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.322493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.323098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] PATH_INFO: `/v3/projects/559ff6b92b3945dfb755141c65f49319/users/837736b344a544a5a14ffda4d4e6762d/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.348599 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=559ff6b92b3945dfb755141c65f49319, user_id=837736b344a544a5a14ffda4d4e6762d, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:55.351612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:55.462546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1369/2741] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:55 2022] PUT /identity/v3/projects/559ff6b92b3945dfb755141c65f49319/users/837736b344a544a5a14ffda4d4e6762d/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:55.467301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:55.479885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3c262-19fa-4a07-bd4d-139afa35f290 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:55.555969 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff498c-74cf-492c-b5ba-5f8ef0cd17d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:55.557033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff498c-74cf-492c-b5ba-5f8ef0cd17d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.557537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff498c-74cf-492c-b5ba-5f8ef0cd17d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.558062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff498c-74cf-492c-b5ba-5f8ef0cd17d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.559296 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff498c-74cf-492c-b5ba-5f8ef0cd17d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:55.562707 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff498c-74cf-492c-b5ba-5f8ef0cd17d6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:55.587957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1373/2742] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:55.593376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:55.604747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16580c49-c864-4917-b9b9-6009e23fbac6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:55.614957 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:55.616205 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.616807 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.617378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] PATH_INFO: `/v3/projects/559ff6b92b3945dfb755141c65f49319/users/837736b344a544a5a14ffda4d4e6762d/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.622266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=559ff6b92b3945dfb755141c65f49319, user_id=837736b344a544a5a14ffda4d4e6762d, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:55.628639 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:55.651223 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1370/2743] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:55 2022] PUT /identity/v3/projects/559ff6b92b3945dfb755141c65f49319/users/837736b344a544a5a14ffda4d4e6762d/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:43:55.665779 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.666380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.666879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.714793 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] MFA Rules not processed for user `837736b344a544a5a14ffda4d4e6762d`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:55.717349 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:55.780682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:55.806535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1374/2744] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:55.825277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:55.829102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:55.924903 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ee9f8-ed87-4e1d-b917-9be7c26a4a02 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:55.930498 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0dda7-47f1-456f-bcd1-2231d502542f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:55.931331 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0dda7-47f1-456f-bcd1-2231d502542f service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:55.931725 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0dda7-47f1-456f-bcd1-2231d502542f service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:55.932128 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0dda7-47f1-456f-bcd1-2231d502542f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:55.949230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0dda7-47f1-456f-bcd1-2231d502542f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:55.952303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0dda7-47f1-456f-bcd1-2231d502542f service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:55.971530 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1371/2745] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:55.995293 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:56.000010 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:56.147896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad270a2-7c39-4605-91c2-3f967e9eed98 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:56.156300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:56.157420 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:56.157874 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:56.158359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] PATH_INFO: `/v3/projects/a6b5152073f748c287990d07c710f023` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:56.160372 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b5152073f748c287990d07c710f023)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:56.163891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:56.410537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b5152073f748c287990d07c710f023', 'request_id': 'req-d07ba5ac-6839-4d49-8f08-009928f285e6'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:56.421696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b5152073f748c287990d07c710f023', 'request_id': 'req-d07ba5ac-6839-4d49-8f08-009928f285e6'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:56.524364 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b5152073f748c287990d07c710f023', 'request_id': 'req-d07ba5ac-6839-4d49-8f08-009928f285e6'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:56.543350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] The token cache is being invalidate because project a6b5152073f748c287990d07c710f023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:56.543350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:56.543350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d07ba5ac-6839-4d49-8f08-009928f285e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:56.545824 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1375/2746] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:55 2022] DELETE /identity/v3/projects/a6b5152073f748c287990d07c710f023 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:56.551568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1881d12-1390-49b1-bcaa-42b657f7f1fd None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:56.551933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1881d12-1390-49b1-bcaa-42b657f7f1fd None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:56.552277 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1881d12-1390-49b1-bcaa-42b657f7f1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:56.576710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d1881d12-1390-49b1-bcaa-42b657f7f1fd None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:56.578217 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d1881d12-1390-49b1-bcaa-42b657f7f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:56.625026 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:56.661233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1372/2747] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:56.672747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4288b1-f302-4e99-b9b8-e01cbf5f8838 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:56.673172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4288b1-f302-4e99-b9b8-e01cbf5f8838 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:56.673578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4288b1-f302-4e99-b9b8-e01cbf5f8838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:56.700706 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7f4288b1-f302-4e99-b9b8-e01cbf5f8838 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:56.702239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f4288b1-f302-4e99-b9b8-e01cbf5f8838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:56.711250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1376/2748] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:56.723667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1881d12-1390-49b1-bcaa-42b657f7f1fd None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:56.735893 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1881d12-1390-49b1-bcaa-42b657f7f1fd None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:56.741730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:56.742403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:56.742738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:56.743009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:56.743683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:43:56.743989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:56.745683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:56.762740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1373/2749] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:43:56.771865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4288b1-f302-4e99-b9b8-e01cbf5f8838 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:56.784555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4288b1-f302-4e99-b9b8-e01cbf5f8838 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:56.791064 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611ead8-e457-4456-98ec-45e8ec324708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:56.791835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ead8-e457-4456-98ec-45e8ec324708 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:56.792235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ead8-e457-4456-98ec-45e8ec324708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:56.792626 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ead8-e457-4456-98ec-45e8ec324708 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:56.793530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611ead8-e457-4456-98ec-45e8ec324708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:56.798701 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611ead8-e457-4456-98ec-45e8ec324708 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:56.986556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1377/2750] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:43:56 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:43:56.992834 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:57.012168 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ef7c-733e-4613-bb97-35f40edabf0d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:57.058422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4783c-480f-4a3f-9a65-8862a45fce01 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:57.059311 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4783c-480f-4a3f-9a65-8862a45fce01 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:57.061106 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4783c-480f-4a3f-9a65-8862a45fce01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:57.100963 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6ce4783c-480f-4a3f-9a65-8862a45fce01 None None] MFA Rules not processed for user `620a40c0e83449a6b39821a589add42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:57.103219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6ce4783c-480f-4a3f-9a65-8862a45fce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:57.115389 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:57.116455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:57.116941 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:57.117442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:57.118603 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:57.123472 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:57.183522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1378/2751] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:57.255330 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1374/2752] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:43:56 2022] POST /identity/v3/users => generated 286 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:43:57.257148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4783c-480f-4a3f-9a65-8862a45fce01 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:57.276006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4783c-480f-4a3f-9a65-8862a45fce01 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:57.283083 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:57.283883 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:57.284464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:57.284721 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:57.285731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:57.288703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:57.305473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1379/2753] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:43:57.310795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:57.311954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:57.324609 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337708d-2b72-4586-9475-cc916533aeac admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:57.330657 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12538050-8ac3-457f-87b0-ccad615e64cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:57.331431 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12538050-8ac3-457f-87b0-ccad615e64cf admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:57.331812 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12538050-8ac3-457f-87b0-ccad615e64cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:57.332129 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12538050-8ac3-457f-87b0-ccad615e64cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:57.332924 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12538050-8ac3-457f-87b0-ccad615e64cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:57.334976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12538050-8ac3-457f-87b0-ccad615e64cf admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:57.349942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1375/2754] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:43:57.354953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:57.370468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0ccd6-9752-413a-8db1-7df4f5db5f14 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:57.381504 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:57.382947 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:57.383515 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:57.384081 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] PATH_INFO: `/v3/projects/d87a7f407bf64c4a9ed59037a569b0a1/users/866e92f9dfb548e1830013af90281362/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:57.411310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87a7f407bf64c4a9ed59037a569b0a1, user_id=866e92f9dfb548e1830013af90281362, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:57.414112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:57.508857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1380/2755] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:43:57 2022] PUT /identity/v3/projects/d87a7f407bf64c4a9ed59037a569b0a1/users/866e92f9dfb548e1830013af90281362/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:57.511056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:57.511733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:57.512306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:57.551506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:57.553895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:57.667686 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1376/2756] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:57.676919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:57.679427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:57.799582 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630df37-d11a-4170-b109-1b29cef3b2ef admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:57.807591 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2ecc3-a42c-4aa4-8377-769bf393e7af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:57.813306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ecc3-a42c-4aa4-8377-769bf393e7af service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:57.813306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ecc3-a42c-4aa4-8377-769bf393e7af service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:57.813306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ecc3-a42c-4aa4-8377-769bf393e7af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:57.821629 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2ecc3-a42c-4aa4-8377-769bf393e7af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:57.823649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2ecc3-a42c-4aa4-8377-769bf393e7af service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:57.844272 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1381/2757] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:43:58.011490 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:58.013974 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:58.036699 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c553652-f268-4dc1-aeec-64a67e3aa2c9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:58.044030 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:58.045662 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:58.050504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:58.051125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] PATH_INFO: `/v3/users/f56a56d562b34e95b8864fc3bc0f6c57` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:58.053140 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56a56d562b34e95b8864fc3bc0f6c57)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:58.061573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:58.337742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:58.347539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56a56d562b34e95b8864fc3bc0f6c57'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:58.349161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56a56d562b34e95b8864fc3bc0f6c57'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:58.349928 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56a56d562b34e95b8864fc3bc0f6c57'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:58.448489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56a56d562b34e95b8864fc3bc0f6c57'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:58.458570 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56a56d562b34e95b8864fc3bc0f6c57'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:58.479233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1377/2758] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:58 2022] DELETE /identity/v3/users/f56a56d562b34e95b8864fc3bc0f6c57 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:58.485307 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:58.485864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:58.485981 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:58.507731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:43:58.508996 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:58.607694 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1382/2759] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:43:58.619084 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:58.621464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:58.735867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972a41b-dcc3-461d-9f96-77228e3438d7 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:58.744583 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:58.745979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:58.752052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:58.752052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:58.766516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:58.769434 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:58.789271 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1378/2760] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:43:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:43:59.362312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:59.362312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:43:59.380990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6110dad-c5b1-4acb-bf3e-fbe7de2cf522 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:59.389673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:59.390761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:59.391074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:59.391596 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] PATH_INFO: `/v3/projects/5d15de97c1c54e06a44d85591005899f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:59.402489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d15de97c1c54e06a44d85591005899f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:59.402489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:43:59.603560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d15de97c1c54e06a44d85591005899f', 'request_id': 'req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:59.611006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:43:59.612371 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d15de97c1c54e06a44d85591005899f', 'request_id': 'req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:59.712597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d15de97c1c54e06a44d85591005899f', 'request_id': 'req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:59.729292 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] The token cache is being invalidate because project 5d15de97c1c54e06a44d85591005899f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:43:59.729651 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:59.730450 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:43:59.733216 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1383/2761] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:43:59 2022] DELETE /identity/v3/projects/5d15de97c1c54e06a44d85591005899f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:43:59.738423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:43:59.750361 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5f0ee-afe2-4901-bdef-2bcf7def8f1c service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:43:59.822364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:43:59.823879 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:43:59.824339 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:43:59.824730 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] PATH_INFO: `/v3/users/837736b344a544a5a14ffda4d4e6762d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:43:59.827045 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837736b344a544a5a14ffda4d4e6762d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:43:59.829555 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:00.066176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837736b344a544a5a14ffda4d4e6762d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:00.067930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837736b344a544a5a14ffda4d4e6762d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:00.069016 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837736b344a544a5a14ffda4d4e6762d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:00.194394 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837736b344a544a5a14ffda4d4e6762d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:00.221276 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6c16e889-104b-40a0-b75f-20ea084bc7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837736b344a544a5a14ffda4d4e6762d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:00.224008 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:00.262266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1379/2762] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:43:59 2022] DELETE /identity/v3/users/837736b344a544a5a14ffda4d4e6762d => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:01.014992 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:01.017974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:01.048864 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e36cf-1ba2-4998-ae35-1f5184404dc9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:01.219250 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:01.220452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:01.220879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:01.221227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] PATH_INFO: `/v3/projects/559ff6b92b3945dfb755141c65f49319` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:01.223913 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=559ff6b92b3945dfb755141c65f49319)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:01.227884 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:01.417099 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '559ff6b92b3945dfb755141c65f49319', 'request_id': 'req-7f6281b6-0ab5-48de-b232-e2a803740b36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:01.437802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '559ff6b92b3945dfb755141c65f49319', 'request_id': 'req-7f6281b6-0ab5-48de-b232-e2a803740b36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:01.616186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '559ff6b92b3945dfb755141c65f49319', 'request_id': 'req-7f6281b6-0ab5-48de-b232-e2a803740b36'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:01.633554 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] The token cache is being invalidate because project 559ff6b92b3945dfb755141c65f49319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:44:01.634055 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:01.636656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f6281b6-0ab5-48de-b232-e2a803740b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:01.636656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1384/2763] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:44:01 2022] DELETE /identity/v3/projects/559ff6b92b3945dfb755141c65f49319 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:01.645710 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03a2bb-5b0f-41e6-aad2-c365c54f20d3 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:01.646324 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03a2bb-5b0f-41e6-aad2-c365c54f20d3 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:01.646695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03a2bb-5b0f-41e6-aad2-c365c54f20d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:01.679518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5c03a2bb-5b0f-41e6-aad2-c365c54f20d3 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:01.684971 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5c03a2bb-5b0f-41e6-aad2-c365c54f20d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:01.792481 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1380/2764] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:01.809136 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f32a1-e43f-403b-bf19-e3da03503598 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:01.810795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f32a1-e43f-403b-bf19-e3da03503598 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:01.811320 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f32a1-e43f-403b-bf19-e3da03503598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:01.845151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a96f32a1-e43f-403b-bf19-e3da03503598 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:01.847606 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a96f32a1-e43f-403b-bf19-e3da03503598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:01.864003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1385/2765] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:01.871505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03a2bb-5b0f-41e6-aad2-c365c54f20d3 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:01.894989 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03a2bb-5b0f-41e6-aad2-c365c54f20d3 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:01.904946 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:01.905947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:01.906480 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:01.907034 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:01.908489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:44:01.909546 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:01.912283 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:01.942127 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1381/2766] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:44:01.943830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f32a1-e43f-403b-bf19-e3da03503598 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:01.961116 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f32a1-e43f-403b-bf19-e3da03503598 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:01.969923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60313804-1c95-412b-a572-1a9381213def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:01.971300 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60313804-1c95-412b-a572-1a9381213def admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:01.971813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60313804-1c95-412b-a572-1a9381213def admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:01.972489 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60313804-1c95-412b-a572-1a9381213def admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:01.973616 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60313804-1c95-412b-a572-1a9381213def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:01.976680 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60313804-1c95-412b-a572-1a9381213def admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:02.102574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1386/2767] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:44:01 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:02.110635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:02.130792 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba19c4-a1db-4658-a6c2-d9d1ad7fd2cd admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:02.279690 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:02.280677 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:02.281086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:02.281559 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:02.282723 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:02.286316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:02.446384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1382/2768] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:44:02 2022] POST /identity/v3/users => generated 292 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:44:02.446652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60313804-1c95-412b-a572-1a9381213def admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:02.467923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60313804-1c95-412b-a572-1a9381213def admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:02.484175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:02.485658 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:02.491532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:02.491532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:02.491532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:02.493722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:02.520039 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1387/2769] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:02.527562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:02.545171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8748ce3-f4e5-46ba-8a84-fbe6bbc3d94b admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:02.560131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64899f-244d-4055-935d-0ddc76744798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:02.560131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64899f-244d-4055-935d-0ddc76744798 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:02.560131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64899f-244d-4055-935d-0ddc76744798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:02.560131 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64899f-244d-4055-935d-0ddc76744798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:02.560707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64899f-244d-4055-935d-0ddc76744798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:02.560707 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64899f-244d-4055-935d-0ddc76744798 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:02.574104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1383/2770] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:02.580716 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:02.595905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc46d35-56df-438a-bb18-0e7b1947058f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:02.603100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:02.603911 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:02.604190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:02.604462 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] PATH_INFO: `/v3/projects/c972d703501a4d18b71cf123930148ae/users/bb05d9eb29fb42f5bde4c4e5049ed5b6/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:02.625477 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c972d703501a4d18b71cf123930148ae, user_id=bb05d9eb29fb42f5bde4c4e5049ed5b6, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:02.628373 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:02.756627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1388/2771] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:02 2022] PUT /identity/v3/projects/c972d703501a4d18b71cf123930148ae/users/bb05d9eb29fb42f5bde4c4e5049ed5b6/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:02.775750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8540-9ae0-4d1f-9164-3fb520429fa9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:02.776679 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8540-9ae0-4d1f-9164-3fb520429fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:02.777035 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8540-9ae0-4d1f-9164-3fb520429fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:02.844064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ffae8540-9ae0-4d1f-9164-3fb520429fa9 None None] MFA Rules not processed for user `bb05d9eb29fb42f5bde4c4e5049ed5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:02.846219 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ffae8540-9ae0-4d1f-9164-3fb520429fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:02.917550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:03.053921 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1384/2772] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:03.061985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:03.097507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:03.202878 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee8e62-40b9-471d-83e9-7545cb70b330 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:03.208026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:03.208887 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:03.209181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:03.209463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:03.210174 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:03.212303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:03.404825 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1389/2773] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:44:03 2022] POST /identity/v3/projects => generated 383 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:03.408576 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae8540-9ae0-4d1f-9164-3fb520429fa9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:03.431002 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae8540-9ae0-4d1f-9164-3fb520429fa9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:03.530765 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:03.531716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:03.532001 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:03.532278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:03.532933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:03.535437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:03.642078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1385/2774] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:44:03 2022] POST /identity/v3/users => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:03.643788 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:03.655894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e874ecf-1c96-4f82-8ba4-04c872502579 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:03.660732 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:03.661398 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:03.661670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:03.661937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:03.662624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:03.664260 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:03.680820 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1390/2775] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:44:03.695403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:03.707579 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad94bf7-19f5-4625-9d7e-f999d67d8702 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:03.713838 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:03.714571 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:03.714845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:03.715113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] PATH_INFO: `/v3/projects/496b83a5daf04cfbbfa389de5ae641cc/users/8c29f0ec1d1143e0bc35a5a9c9745b81/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:03.736648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496b83a5daf04cfbbfa389de5ae641cc, user_id=8c29f0ec1d1143e0bc35a5a9c9745b81, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:03.739510 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:03.833567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1386/2776] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:03 2022] PUT /identity/v3/projects/496b83a5daf04cfbbfa389de5ae641cc/users/8c29f0ec1d1143e0bc35a5a9c9745b81/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:44:03.838212 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:03.851953 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4637a-9ffa-4d1f-873e-3aeae5bfabe4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:03.941188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cbdcc-3c35-4b6a-88fe-2b8febdc5bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:03.942273 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cbdcc-3c35-4b6a-88fe-2b8febdc5bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:03.942708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cbdcc-3c35-4b6a-88fe-2b8febdc5bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:03.943163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cbdcc-3c35-4b6a-88fe-2b8febdc5bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:03.944510 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cbdcc-3c35-4b6a-88fe-2b8febdc5bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:03.947961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cbdcc-3c35-4b6a-88fe-2b8febdc5bc5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:03.971084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1391/2777] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:03.976955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:04.003567 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042338-9517-43e4-8eee-2673e5d8b145 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:04.015442 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:04.018294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:04.019211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:04.020418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] PATH_INFO: `/v3/projects/496b83a5daf04cfbbfa389de5ae641cc/users/8c29f0ec1d1143e0bc35a5a9c9745b81/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:04.028975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496b83a5daf04cfbbfa389de5ae641cc, user_id=8c29f0ec1d1143e0bc35a5a9c9745b81, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:04.038248 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:04.111069 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1387/2778] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:03 2022] PUT /identity/v3/projects/496b83a5daf04cfbbfa389de5ae641cc/users/8c29f0ec1d1143e0bc35a5a9c9745b81/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:04.215695 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36661b45-de81-446f-b014-e192d0980b63 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:04.219738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36661b45-de81-446f-b014-e192d0980b63 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:04.219738 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36661b45-de81-446f-b014-e192d0980b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:04.311567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-36661b45-de81-446f-b014-e192d0980b63 None None] MFA Rules not processed for user `8c29f0ec1d1143e0bc35a5a9c9745b81`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:04.311567 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-36661b45-de81-446f-b014-e192d0980b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:04.531759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1392/2779] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:04.547538 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:04.550464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:04.719436 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e10e7-c2dc-4d34-a40a-f62e411672a9 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:04.737206 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:04.737867 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:04.738210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:04.738523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:04.813379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:04.816252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:04.860201 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1388/2780] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:05.123119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36661b45-de81-446f-b014-e192d0980b63 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:05.155461 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36661b45-de81-446f-b014-e192d0980b63 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:05.161640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abebfc1-ee32-4dac-a55a-55628e741463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:05.162622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abebfc1-ee32-4dac-a55a-55628e741463 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:05.163124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abebfc1-ee32-4dac-a55a-55628e741463 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:05.163628 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abebfc1-ee32-4dac-a55a-55628e741463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:05.175990 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:05.300970 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abebfc1-ee32-4dac-a55a-55628e741463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:05.303670 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abebfc1-ee32-4dac-a55a-55628e741463 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:05.322236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1393/2781] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:06.170210 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:06.235727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:06.288120 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace55c1c-0e1e-432a-b450-2c43bafad5c7 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:06.297257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:06.297929 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:06.298917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:06.298917 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] PATH_INFO: `/v3/users/bb05d9eb29fb42f5bde4c4e5049ed5b6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:06.300198 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb05d9eb29fb42f5bde4c4e5049ed5b6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:06.304996 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:06.564391 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb05d9eb29fb42f5bde4c4e5049ed5b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:06.567012 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb05d9eb29fb42f5bde4c4e5049ed5b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:06.568316 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb05d9eb29fb42f5bde4c4e5049ed5b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:06.734188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb05d9eb29fb42f5bde4c4e5049ed5b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:06.745309 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb05d9eb29fb42f5bde4c4e5049ed5b6'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:06.806256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1389/2782] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:44:06 2022] DELETE /identity/v3/users/bb05d9eb29fb42f5bde4c4e5049ed5b6 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:06.813358 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f26d8-7233-4b34-9bf7-fec3631a5c6d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:06.813933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f26d8-7233-4b34-9bf7-fec3631a5c6d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:06.814332 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f26d8-7233-4b34-9bf7-fec3631a5c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:06.846072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4f0f26d8-7233-4b34-9bf7-fec3631a5c6d None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:06.848583 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4f0f26d8-7233-4b34-9bf7-fec3631a5c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:06.958662 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1394/2783] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:06.973742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:06.976829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:07.112028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfbcc9-2179-468f-a286-d910e43132ef admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:07.116940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:07.117909 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:07.118295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:07.118685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:07.133876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:07.135798 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:07.150364 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1390/2784] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:44:07.449205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee9e88-8bc1-4341-aaf0-4f3569d11e37 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:07.449487 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee9e88-8bc1-4341-aaf0-4f3569d11e37 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:07.450652 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee9e88-8bc1-4341-aaf0-4f3569d11e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:07.490227 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-adee9e88-8bc1-4341-aaf0-4f3569d11e37 None None] MFA Rules not processed for user `866e92f9dfb548e1830013af90281362`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:07.491493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-adee9e88-8bc1-4341-aaf0-4f3569d11e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:07.591240 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1395/2785] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:07.597329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:07.598848 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:07.620661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea78c96-24c1-49b1-bc0f-07259e86e6b7 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:07.646351 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:07.647554 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:07.648449 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:07.649245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:07.650711 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:07.655916 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:07.800881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1391/2786] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:44:07 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:07.803274 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee9e88-8bc1-4341-aaf0-4f3569d11e37 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:07.823997 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee9e88-8bc1-4341-aaf0-4f3569d11e37 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:07.889061 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:07.925713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:07.928368 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:07.960964 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:07.965040 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:07.967236 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:07.967989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:07.969848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:07.979903 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:08.011920 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:08.057229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7d95-5aa2-4b7a-ada8-cfbbe65d7551 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:08.063174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:08.064419 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:08.064877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:08.065336 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] PATH_INFO: `/v3/projects/c972d703501a4d18b71cf123930148ae` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:08.073493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c972d703501a4d18b71cf123930148ae)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:08.077109 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:08.167089 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1396/2787] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:44:07 2022] POST /identity/v3/users => generated 292 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:08.173507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:08.190095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b38724-860d-4758-b5ea-f759ae3917b6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:08.201915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:08.203148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:08.203641 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:08.204137 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:08.205366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:08.211915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:08.230374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1397/2788] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:44:08.237488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:08.256045 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfe98b-8d39-4236-8b8d-1e59295958ac admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:08.267863 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c972d703501a4d18b71cf123930148ae', 'request_id': 'req-6cbfc8a5-ed20-4562-ad98-7365892c7e88'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.278549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c972d703501a4d18b71cf123930148ae', 'request_id': 'req-6cbfc8a5-ed20-4562-ad98-7365892c7e88'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.286722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c972d703501a4d18b71cf123930148ae', 'request_id': 'req-6cbfc8a5-ed20-4562-ad98-7365892c7e88'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.346407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:08.347593 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:08.347871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:08.348172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] PATH_INFO: `/v3/projects/777cde9a912c4bc8999db494aae5bece/users/3c041e10d4a34cecb5638cca1898675d/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:08.368166 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777cde9a912c4bc8999db494aae5bece, user_id=3c041e10d4a34cecb5638cca1898675d, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:08.371122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:08.437453 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] The token cache is being invalidate because project c972d703501a4d18b71cf123930148ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:44:08.437766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.438748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.440636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1392/2789] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:44:07 2022] DELETE /identity/v3/projects/c972d703501a4d18b71cf123930148ae => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:44:08.445108 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:08.446736 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:08.467301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfc8a5-ed20-4562-ad98-7365892c7e88 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:08.478435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:08.478435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:08.478435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:08.478435 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] PATH_INFO: `/v3/users/8c29f0ec1d1143e0bc35a5a9c9745b81` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:08.479280 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c29f0ec1d1143e0bc35a5a9c9745b81)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:08.483681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:08.565149 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1398/2790] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:08 2022] PUT /identity/v3/projects/777cde9a912c4bc8999db494aae5bece/users/3c041e10d4a34cecb5638cca1898675d/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:44:08.566773 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:08.569613 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:08.669522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45abd6-aa73-435b-b2d3-863e6873acd4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:08.681114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:08.682572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:08.683044 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:08.683677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:08.694656 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:08.696333 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:08.714393 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1399/2791] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:08.723015 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:08.736841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb87579-2b58-4887-98c1-c596b9653574 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:08.742389 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37482064-d147-4587-816f-3509cf942708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:08.743157 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482064-d147-4587-816f-3509cf942708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:08.743436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482064-d147-4587-816f-3509cf942708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:08.743761 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482064-d147-4587-816f-3509cf942708 admin admin] PATH_INFO: `/v3/projects/777cde9a912c4bc8999db494aae5bece/users/3c041e10d4a34cecb5638cca1898675d/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:08.747158 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37482064-d147-4587-816f-3509cf942708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777cde9a912c4bc8999db494aae5bece, user_id=3c041e10d4a34cecb5638cca1898675d, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:08.750474 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37482064-d147-4587-816f-3509cf942708 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:08.767848 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1400/2792] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:08 2022] PUT /identity/v3/projects/777cde9a912c4bc8999db494aae5bece/users/3c041e10d4a34cecb5638cca1898675d/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:08.780017 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c29f0ec1d1143e0bc35a5a9c9745b81'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.781478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c29f0ec1d1143e0bc35a5a9c9745b81'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.793586 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c29f0ec1d1143e0bc35a5a9c9745b81'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.936266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c29f0ec1d1143e0bc35a5a9c9745b81'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.947437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5ea89985-e6a0-4d8c-920c-4bbd502021e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c29f0ec1d1143e0bc35a5a9c9745b81'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:08.979142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1393/2793] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:44:08 2022] DELETE /identity/v3/users/8c29f0ec1d1143e0bc35a5a9c9745b81 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:44:09.647900 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37482064-d147-4587-816f-3509cf942708 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:09.652408 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-37482064-d147-4587-816f-3509cf942708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:09.725830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:09.815445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37482064-d147-4587-816f-3509cf942708 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:09.945030 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:09.946252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:09.947123 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:09.947252 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] PATH_INFO: `/v3/projects/496b83a5daf04cfbbfa389de5ae641cc` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:09.949363 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=496b83a5daf04cfbbfa389de5ae641cc)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:09.953415 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:10.151445 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496b83a5daf04cfbbfa389de5ae641cc', 'request_id': 'req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:10.160972 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496b83a5daf04cfbbfa389de5ae641cc', 'request_id': 'req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:10.238463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496b83a5daf04cfbbfa389de5ae641cc', 'request_id': 'req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:10.255295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] The token cache is being invalidate because project 496b83a5daf04cfbbfa389de5ae641cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:44:10.255754 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:10.256829 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f1db96d-3fe4-4c41-8cc2-23c8c118ae5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:10.262306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1401/2794] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:44:09 2022] DELETE /identity/v3/projects/496b83a5daf04cfbbfa389de5ae641cc => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:10.264252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d01de1-28ab-49c7-baf2-9511e49da92c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:10.264827 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d01de1-28ab-49c7-baf2-9511e49da92c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:10.264988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d01de1-28ab-49c7-baf2-9511e49da92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:10.290584 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-95d01de1-28ab-49c7-baf2-9511e49da92c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:10.291845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-95d01de1-28ab-49c7-baf2-9511e49da92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:10.334187 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:10.438847 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1394/2795] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:10.445631 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dd3e1-3d60-4b41-9b3f-7b159962e0a2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:10.446117 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dd3e1-3d60-4b41-9b3f-7b159962e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:10.446574 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dd3e1-3d60-4b41-9b3f-7b159962e0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:10.487442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6a5dd3e1-3d60-4b41-9b3f-7b159962e0a2 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:10.489637 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6a5dd3e1-3d60-4b41-9b3f-7b159962e0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:10.515098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1402/2796] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:10.515372 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d01de1-28ab-49c7-baf2-9511e49da92c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:10.526726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d01de1-28ab-49c7-baf2-9511e49da92c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:10.532615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:10.533385 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:10.535342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:10.535669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:10.538096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:44:10.538096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:10.538463 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:10.554235 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1395/2797] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:44:10.560543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dd3e1-3d60-4b41-9b3f-7b159962e0a2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:10.571635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dd3e1-3d60-4b41-9b3f-7b159962e0a2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:10.577555 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:10.578374 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:10.578802 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:10.579323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:10.580187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:10.582577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:10.885925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1403/2798] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:44:10 2022] POST /identity/v3/projects => generated 377 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:10.891742 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:10.903523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff08a5-9c64-4b9f-9b73-4da4a725c81d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:11.000648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:11.001400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:11.001684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:11.001954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:11.002801 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:11.005814 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:11.124102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1396/2799] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:44:10 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:11.129484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:11.144813 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c364-73b9-44d5-8d49-6f1829ecc474 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:11.154535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:11.156710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:11.156710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:11.156710 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:11.161975 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:11.161975 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:11.177945 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1404/2800] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:11.182363 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:11.200269 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d182da-3ef9-4f38-be13-2895733e45a1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:11.211955 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2676e-dde4-40fd-bcc9-e85ad5d1a3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:11.213658 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2676e-dde4-40fd-bcc9-e85ad5d1a3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:11.214241 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2676e-dde4-40fd-bcc9-e85ad5d1a3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:11.214753 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2676e-dde4-40fd-bcc9-e85ad5d1a3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:11.216194 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2676e-dde4-40fd-bcc9-e85ad5d1a3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:11.221911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2676e-dde4-40fd-bcc9-e85ad5d1a3ec admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:11.247456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1397/2801] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:11.254151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:11.268814 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11dfcf-c869-4c50-92ae-6aea80d55f91 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:11.275237 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:11.276318 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:11.276609 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:11.276885 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] PATH_INFO: `/v3/projects/4b04e565d4e94f6883cd6105f4ddd7ad/users/dec7399a0d764792a719611ebf9ec0a1/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:11.307853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b04e565d4e94f6883cd6105f4ddd7ad, user_id=dec7399a0d764792a719611ebf9ec0a1, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:11.307853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:11.519160 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1405/2802] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:11 2022] PUT /identity/v3/projects/4b04e565d4e94f6883cd6105f4ddd7ad/users/dec7399a0d764792a719611ebf9ec0a1/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:11.532985 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a105035-d8d8-4132-b0ba-0480234652ec None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:11.533645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a105035-d8d8-4132-b0ba-0480234652ec None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:11.534225 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a105035-d8d8-4132-b0ba-0480234652ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:11.585122 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-1a105035-d8d8-4132-b0ba-0480234652ec None None] MFA Rules not processed for user `dec7399a0d764792a719611ebf9ec0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:11.586889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1a105035-d8d8-4132-b0ba-0480234652ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:11.719824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1398/2803] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:44:11.725614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:11.791869 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:11.849979 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e264d2b-965e-4dd8-ba74-049b773c7f99 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:11.858638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:11.858638 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:11.859024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:11.859235 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:11.860205 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:11.861866 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:12.048556 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1406/2804] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:44:11 2022] POST /identity/v3/projects => generated 377 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:12.054171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a105035-d8d8-4132-b0ba-0480234652ec None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:12.071933 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a105035-d8d8-4132-b0ba-0480234652ec None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:12.212305 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:12.214052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:12.214944 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:12.216104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:12.219378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:12.224278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:12.287829 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:12.360778 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1399/2805] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:44:12 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:44:12.369678 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:12.409036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413d3a2-eb8e-4916-bb03-82410c0aba37 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:12.422343 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:12.424023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:12.425063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:12.425786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:12.428067 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:12.432350 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:12.470807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1407/2806] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:12.477036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:12.508608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23469e3a-76e6-4063-83c4-e01a9cacb94e admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:12.523239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e610ce-d707-4777-918c-550aa4252029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:12.534945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e610ce-d707-4777-918c-550aa4252029 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:12.538052 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e610ce-d707-4777-918c-550aa4252029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:12.538684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e610ce-d707-4777-918c-550aa4252029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:12.540876 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e610ce-d707-4777-918c-550aa4252029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:12.551384 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e610ce-d707-4777-918c-550aa4252029 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:12.580922 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1400/2807] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:12.588431 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:12.627987 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07edd0-d691-4356-be9e-46157a0ca443 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:12.638119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:12.644380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:12.644380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:12.644380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] PATH_INFO: `/v3/projects/637b342d2ad44841abb492845fcc938c/users/3219806a25d84aceace1e2db4d94552d/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:12.708251 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=637b342d2ad44841abb492845fcc938c, user_id=3219806a25d84aceace1e2db4d94552d, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:12.711465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:12.863853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1408/2808] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:12 2022] PUT /identity/v3/projects/637b342d2ad44841abb492845fcc938c/users/3219806a25d84aceace1e2db4d94552d/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:12.886423 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ab72-e333-4245-a13d-cce6e16ae7e4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:12.895976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ab72-e333-4245-a13d-cce6e16ae7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:12.895976 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ab72-e333-4245-a13d-cce6e16ae7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:12.953529 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-eb99ab72-e333-4245-a13d-cce6e16ae7e4 None None] MFA Rules not processed for user `3219806a25d84aceace1e2db4d94552d`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:12.960301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-eb99ab72-e333-4245-a13d-cce6e16ae7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:13.093925 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:13.097303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1401/2809] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:13.249341 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aa3e6-1c83-4968-9f01-2469d2d2a59b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:13.257693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:13.258879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:13.259241 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:13.259612 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:13.260440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:13.264207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:13.445301 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1409/2810] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:44:13 2022] POST /identity/v3/projects => generated 377 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:44:13.450015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ab72-e333-4245-a13d-cce6e16ae7e4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:13.473500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ab72-e333-4245-a13d-cce6e16ae7e4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:13.594902 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:13.596105 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:13.596604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:13.597029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:13.598216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:13.602238 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:13.735376 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1402/2811] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:44:13 2022] POST /identity/v3/users => generated 295 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:13.747200 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:13.753650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a23d4d-6e6e-4881-95ac-6f2881fbd401 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:13.759334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:13.759985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:13.760264 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:13.760538 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:13.761214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:13.764312 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:13.780457 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1410/2812] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:44:13.788715 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:13.807285 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacbec-3279-4bb0-a0f0-33b82f152550 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:13.815595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:13.816558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:13.817021 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:13.817487 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] PATH_INFO: `/v3/projects/32a121a605594fdaac205a00dfafcb2f/users/385bd9371a9c43378c85f14a38613556/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:13.844009 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a121a605594fdaac205a00dfafcb2f, user_id=385bd9371a9c43378c85f14a38613556, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:13.848688 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:13.986660 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1403/2813] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:13 2022] PUT /identity/v3/projects/32a121a605594fdaac205a00dfafcb2f/users/385bd9371a9c43378c85f14a38613556/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:44:13.993060 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:14.014896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaca40-7704-45db-9c60-54777b64ef5e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:14.041026 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:14.131782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae5797-e684-4873-8881-dd5e6c65b532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:14.134422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae5797-e684-4873-8881-dd5e6c65b532 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:14.134422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae5797-e684-4873-8881-dd5e6c65b532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:14.134422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae5797-e684-4873-8881-dd5e6c65b532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:14.134939 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae5797-e684-4873-8881-dd5e6c65b532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:14.136309 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae5797-e684-4873-8881-dd5e6c65b532 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:14.157765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1411/2814] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:14.160616 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:14.169871 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:14.182041 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32fefb-a96c-4d52-8af6-0e3ad72c025a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:14.191764 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:14.194216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:14.194630 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:14.195053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] PATH_INFO: `/v3/projects/32a121a605594fdaac205a00dfafcb2f/users/385bd9371a9c43378c85f14a38613556/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:14.198574 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a121a605594fdaac205a00dfafcb2f, user_id=385bd9371a9c43378c85f14a38613556, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:14.205091 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:14.226218 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1404/2815] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:14 2022] PUT /identity/v3/projects/32a121a605594fdaac205a00dfafcb2f/users/385bd9371a9c43378c85f14a38613556/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:14.248386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85886ee8-def9-471b-b7b0-12a0c6f827ac None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:14.248776 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85886ee8-def9-471b-b7b0-12a0c6f827ac None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:14.249907 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85886ee8-def9-471b-b7b0-12a0c6f827ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:14.281855 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-85886ee8-def9-471b-b7b0-12a0c6f827ac None None] MFA Rules not processed for user `385bd9371a9c43378c85f14a38613556`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:14.283258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-85886ee8-def9-471b-b7b0-12a0c6f827ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:14.403952 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1412/2816] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:14.416298 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:14.418311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:14.490738 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c72a9-46a3-4c8b-887d-68c680c0a0ce admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:14.497748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8b9ae-4c9e-47e2-8032-2f3f250dcbee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:14.499047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b9ae-4c9e-47e2-8032-2f3f250dcbee service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:14.499654 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b9ae-4c9e-47e2-8032-2f3f250dcbee service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:14.500189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8b9ae-4c9e-47e2-8032-2f3f250dcbee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:14.620499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8b9ae-4c9e-47e2-8032-2f3f250dcbee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:14.623667 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8b9ae-4c9e-47e2-8032-2f3f250dcbee service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:14.636456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1405/2817] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:44:17.242172 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85886ee8-def9-471b-b7b0-12a0c6f827ac None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:17.258559 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85886ee8-def9-471b-b7b0-12a0c6f827ac None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:17.273228 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:17.275085 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:17.275703 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:17.276421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:17.447366 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:17.448895 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:17.477910 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1413/2818] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:17.722565 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e5c1-0658-498b-b2ed-e6bf86f9c4c9 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:17.725524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e5c1-0658-498b-b2ed-e6bf86f9c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:17.725524 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e5c1-0658-498b-b2ed-e6bf86f9c4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:17.779086 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-68f0e5c1-0658-498b-b2ed-e6bf86f9c4c9 None None] MFA Rules not processed for user `3c041e10d4a34cecb5638cca1898675d`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:17.781234 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-68f0e5c1-0658-498b-b2ed-e6bf86f9c4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:17.932516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1406/2819] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:17.958306 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:17.960698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:18.030858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:18.131752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e09850-fbaf-48f3-b917-decd18f3b871 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:18.138496 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b855-734d-4f06-b6cc-f5d84542c697 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:18.139130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b855-734d-4f06-b6cc-f5d84542c697 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:18.139407 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b855-734d-4f06-b6cc-f5d84542c697 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:18.139677 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b855-734d-4f06-b6cc-f5d84542c697 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:18.141625 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52a0b855-734d-4f06-b6cc-f5d84542c697 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:18.234098 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b855-734d-4f06-b6cc-f5d84542c697 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:18.236033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b855-734d-4f06-b6cc-f5d84542c697 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:18.248138 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1414/2820] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:19.891333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0e5c1-0658-498b-b2ed-e6bf86f9c4c9 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:19.910587 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0e5c1-0658-498b-b2ed-e6bf86f9c4c9 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:19.947289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:19.948796 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:19.949278 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:19.951125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:20.008180 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:20.014515 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:20.046099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1407/2821] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:21.528159 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4d61-d219-46a5-9f16-83b9d3ad29eb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:21.529031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4d61-d219-46a5-9f16-83b9d3ad29eb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:21.529197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4d61-d219-46a5-9f16-83b9d3ad29eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:21.570985 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2b7c4d61-d219-46a5-9f16-83b9d3ad29eb None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:21.572778 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2b7c4d61-d219-46a5-9f16-83b9d3ad29eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:21.734592 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1415/2822] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:21.956685 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:21.960142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:22.013605 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:22.111684 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544987-b261-4ebe-aade-d5d6e57caf9c service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:22.118082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe8cd3-eaff-4730-8f41-7e974ca7252e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:22.118722 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe8cd3-eaff-4730-8f41-7e974ca7252e service swift] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:22.119158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe8cd3-eaff-4730-8f41-7e974ca7252e service swift] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:22.119437 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe8cd3-eaff-4730-8f41-7e974ca7252e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:22.131015 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe8cd3-eaff-4730-8f41-7e974ca7252e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:22.134830 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe8cd3-eaff-4730-8f41-7e974ca7252e service swift] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:22.148520 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1408/2823] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:44:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:44:22.641931 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454087-1363-4ab8-a855-3f245235798f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:22.643155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454087-1363-4ab8-a855-3f245235798f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:22.643155 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454087-1363-4ab8-a855-3f245235798f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:22.678058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e9454087-1363-4ab8-a855-3f245235798f None None] MFA Rules not processed for user `866e92f9dfb548e1830013af90281362`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:22.678058 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e9454087-1363-4ab8-a855-3f245235798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:22.679210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1416/2824] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:23.577823 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582ad9-b6ce-488e-900d-de33bbd829f7 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:23.578451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582ad9-b6ce-488e-900d-de33bbd829f7 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:23.578451 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582ad9-b6ce-488e-900d-de33bbd829f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:23.602906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-a9582ad9-b6ce-488e-900d-de33bbd829f7 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:23.604169 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a9582ad9-b6ce-488e-900d-de33bbd829f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:23.613414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1409/2825] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 24 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:23.628440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9454087-1363-4ab8-a855-3f245235798f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:23.651782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9454087-1363-4ab8-a855-3f245235798f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:23.662622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:23.662622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:23.662622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:23.662622 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:23.677446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:23.679310 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:23.691839 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1417/2826] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:44:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:44:24.418110 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcb0b3-4a3f-48a4-8230-8c930467e334 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:24.418404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcb0b3-4a3f-48a4-8230-8c930467e334 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:24.418678 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcb0b3-4a3f-48a4-8230-8c930467e334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:24.440740 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-45dcb0b3-4a3f-48a4-8230-8c930467e334 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:24.442087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-45dcb0b3-4a3f-48a4-8230-8c930467e334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:24.453265 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1410/2827] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 24 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:24.470584 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:24.500534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a70cfe-3898-48c7-8c5f-ac3f70a93502 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:24.513876 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:24.514975 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:24.515438 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:24.515914 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:24.530649 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:24.532560 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:24.544518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1418/2828] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:44:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 14:44:25.353824 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed144323-c2f6-4090-9aa4-88aa95964157 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:25.354516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed144323-c2f6-4090-9aa4-88aa95964157 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:25.354516 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed144323-c2f6-4090-9aa4-88aa95964157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:25.379471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-ed144323-c2f6-4090-9aa4-88aa95964157 None None] MFA Rules not processed for user `a9c450c65b6849feb8c4250a1ca09fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:25.381058 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-ed144323-c2f6-4090-9aa4-88aa95964157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:25.389564 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1411/2829] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 24 14:44:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:25.404334 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:25.416942 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928844-ca7c-4058-9a13-f4e321d44116 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:25.423922 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5760ca-43b1-420d-9844-a1e12cde21d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c4eeb768c04233bcbfc786b3ba20b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:25.424627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5760ca-43b1-420d-9844-a1e12cde21d7 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:25.424905 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5760ca-43b1-420d-9844-a1e12cde21d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:25.426498 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5760ca-43b1-420d-9844-a1e12cde21d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:25.436840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5760ca-43b1-420d-9844-a1e12cde21d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:25.438543 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5760ca-43b1-420d-9844-a1e12cde21d7 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:25.448217 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1419/2830] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 24 14:44:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 14:44:25.645077 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed144323-c2f6-4090-9aa4-88aa95964157 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:25.727840 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed144323-c2f6-4090-9aa4-88aa95964157 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:25.733718 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:25.734408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:25.734701 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:25.734967 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] PATH_INFO: `/v3/users/dec7399a0d764792a719611ebf9ec0a1` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:25.736231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec7399a0d764792a719611ebf9ec0a1)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:25.738220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:25.969918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec7399a0d764792a719611ebf9ec0a1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:25.970896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec7399a0d764792a719611ebf9ec0a1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:25.971786 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec7399a0d764792a719611ebf9ec0a1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:26.055078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec7399a0d764792a719611ebf9ec0a1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:26.067904 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec7399a0d764792a719611ebf9ec0a1'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:26.112454 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1412/2831] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:44:25 2022] DELETE /identity/v3/users/dec7399a0d764792a719611ebf9ec0a1 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:26.118427 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:26.118924 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:26.119413 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:26.153572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:26.166016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:26.166016 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:26.248518 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1420/2832] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:26.270611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:26.270611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:26.276203 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:26.357594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902e46-dbd7-455d-806a-404b506ec867 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:26.361899 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:26.362558 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:26.363079 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:26.363766 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:26.385347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:26.388698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:26.407875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1413/2833] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:26.980434 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:26.982758 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:27.004974 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8276ff7-6f6c-4195-9b90-694963161dc8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:27.015954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:27.016978 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:27.017390 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:27.018119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] PATH_INFO: `/v3/projects/4b04e565d4e94f6883cd6105f4ddd7ad` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:27.020830 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b04e565d4e94f6883cd6105f4ddd7ad)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:27.024870 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:27.266572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b04e565d4e94f6883cd6105f4ddd7ad', 'request_id': 'req-d1464a00-0a71-4813-a229-55ef0cda9ba1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.282941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b04e565d4e94f6883cd6105f4ddd7ad', 'request_id': 'req-d1464a00-0a71-4813-a229-55ef0cda9ba1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.415114 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b04e565d4e94f6883cd6105f4ddd7ad', 'request_id': 'req-d1464a00-0a71-4813-a229-55ef0cda9ba1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.439635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] The token cache is being invalidate because project 4b04e565d4e94f6883cd6105f4ddd7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:44:27.440169 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.441561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.443422 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1421/2834] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:44:26 2022] DELETE /identity/v3/projects/4b04e565d4e94f6883cd6105f4ddd7ad => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:27.447772 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:27.459757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039b567-2acc-4a54-8424-c73f6e9708b4 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:27.525189 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:27.525716 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:27.526018 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:27.526296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] PATH_INFO: `/v3/users/3219806a25d84aceace1e2db4d94552d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:27.527526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3219806a25d84aceace1e2db4d94552d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:27.529503 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:27.749303 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3219806a25d84aceace1e2db4d94552d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.750039 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3219806a25d84aceace1e2db4d94552d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.750880 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3219806a25d84aceace1e2db4d94552d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.889247 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3219806a25d84aceace1e2db4d94552d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.899022 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3219806a25d84aceace1e2db4d94552d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:27.921499 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1414/2835] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:44:27 2022] DELETE /identity/v3/users/3219806a25d84aceace1e2db4d94552d => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:44:28.515238 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:28.517075 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:28.536810 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1464a00-0a71-4813-a229-55ef0cda9ba1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:28.632562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:28.633615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:28.634148 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:28.634633 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] PATH_INFO: `/v3/projects/637b342d2ad44841abb492845fcc938c` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:28.637207 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=637b342d2ad44841abb492845fcc938c)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:28.641535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:28.709721 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:28.712261 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:28.733911 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da708067-01dc-49a6-a771-a2615a5396a6 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:28.739230 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:28.739884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:28.740162 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:28.740440 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] PATH_INFO: `/v3/users/620a40c0e83449a6b39821a589add42c` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:28.741695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] RBAC: Authorizing `identity:delete_user(user_id=620a40c0e83449a6b39821a589add42c)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:28.743706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:28.773535 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:28.848465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:28.873231 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637b342d2ad44841abb492845fcc938c', 'request_id': 'req-b867cc29-4508-412b-963d-b21347adb013'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.883501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637b342d2ad44841abb492845fcc938c', 'request_id': 'req-b867cc29-4508-412b-963d-b21347adb013'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.952104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637b342d2ad44841abb492845fcc938c', 'request_id': 'req-b867cc29-4508-412b-963d-b21347adb013'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.969960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] The token cache is being invalidate because project 637b342d2ad44841abb492845fcc938c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:44:28.970400 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.971429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.973239 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620a40c0e83449a6b39821a589add42c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.974084 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1422/2836] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:44:28 2022] DELETE /identity/v3/projects/637b342d2ad44841abb492845fcc938c => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:28.974615 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620a40c0e83449a6b39821a589add42c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.975756 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620a40c0e83449a6b39821a589add42c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:28.979095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:28.981303 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:29.044096 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620a40c0e83449a6b39821a589add42c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.055988 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-7e7118ad-c9a9-4498-b00e-5dce5fcbd100 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620a40c0e83449a6b39821a589add42c'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.064868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867cc29-4508-412b-963d-b21347adb013 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:29.071901 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:29.073119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:29.073539 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:29.073834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] PATH_INFO: `/v3/users/385bd9371a9c43378c85f14a38613556` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:29.076615 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=385bd9371a9c43378c85f14a38613556)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:29.080681 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:29.104645 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1415/2837] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:44:28 2022] DELETE /identity/v3/users/620a40c0e83449a6b39821a589add42c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:29.112430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:29.115287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:29.115287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:29.134739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:29.136326 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:29.217767 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1416/2838] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:44:29.228051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:29.229573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:29.304784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa40b-2ada-4da5-85a6-5cdd373fdb2e None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:29.310557 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '385bd9371a9c43378c85f14a38613556'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.310744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '385bd9371a9c43378c85f14a38613556'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.311005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:29.311954 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:29.312613 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:29.313242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:29.322112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '385bd9371a9c43378c85f14a38613556'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.323032 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '385bd9371a9c43378c85f14a38613556'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.329749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:29.331287 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:29.342820 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1417/2839] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:44:29.405125 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '385bd9371a9c43378c85f14a38613556'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.439927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1423/2840] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:44:28 2022] DELETE /identity/v3/users/385bd9371a9c43378c85f14a38613556 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:29.704691 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:29.707312 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:29.724473 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7efe72-0a84-4562-9aba-251c30b26215 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:29.785761 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:29.820430 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:29.821089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:29.821366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:29.821641 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] PATH_INFO: `/v3/projects/5b24f59c5d7a45cbb034535eba660e7a` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:29.822901 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b24f59c5d7a45cbb034535eba660e7a)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:29.827262 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:29.953562 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b24f59c5d7a45cbb034535eba660e7a', 'request_id': 'req-d141fd4c-ecde-49e0-89ce-21675e06bd9d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.963455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b24f59c5d7a45cbb034535eba660e7a', 'request_id': 'req-d141fd4c-ecde-49e0-89ce-21675e06bd9d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:29.971504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b24f59c5d7a45cbb034535eba660e7a', 'request_id': 'req-d141fd4c-ecde-49e0-89ce-21675e06bd9d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.051418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:30.054100 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:30.067733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] The token cache is being invalidate because project 5b24f59c5d7a45cbb034535eba660e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:44:30.067733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.067733 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-d141fd4c-ecde-49e0-89ce-21675e06bd9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.069746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1418/2841] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:44:29 2022] DELETE /identity/v3/projects/5b24f59c5d7a45cbb034535eba660e7a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:30.086822 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35797f-3ab3-41f0-ac34-083599f092c4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:30.172168 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:30.200506 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:30.201379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:30.201749 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:30.202133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] PATH_INFO: `/v3/projects/32a121a605594fdaac205a00dfafcb2f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:30.203586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a121a605594fdaac205a00dfafcb2f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:30.205719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:30.376699 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a121a605594fdaac205a00dfafcb2f', 'request_id': 'req-28a147f3-58f4-47ce-95f7-14a9a585d75b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.387578 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a121a605594fdaac205a00dfafcb2f', 'request_id': 'req-28a147f3-58f4-47ce-95f7-14a9a585d75b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.501795 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a121a605594fdaac205a00dfafcb2f', 'request_id': 'req-28a147f3-58f4-47ce-95f7-14a9a585d75b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.516190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] The token cache is being invalidate because project 32a121a605594fdaac205a00dfafcb2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:44:30.516190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.516190 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28a147f3-58f4-47ce-95f7-14a9a585d75b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:44:30.516865 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1424/2842] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:44:30 2022] DELETE /identity/v3/projects/32a121a605594fdaac205a00dfafcb2f => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:44:30.521906 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151513d-3c97-4564-86a8-4959d4984643 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:30.522228 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151513d-3c97-4564-86a8-4959d4984643 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:30.522506 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151513d-3c97-4564-86a8-4959d4984643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:30.545885 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-5151513d-3c97-4564-86a8-4959d4984643 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:30.547176 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5151513d-3c97-4564-86a8-4959d4984643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:30.624087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1419/2843] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:30.634180 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aac7d-3e42-44b0-8415-9da1cdbacc87 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:30.634465 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aac7d-3e42-44b0-8415-9da1cdbacc87 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:30.634743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aac7d-3e42-44b0-8415-9da1cdbacc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:30.659698 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6e9aac7d-3e42-44b0-8415-9da1cdbacc87 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:30.660951 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6e9aac7d-3e42-44b0-8415-9da1cdbacc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:30.669655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1425/2844] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:30.676790 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151513d-3c97-4564-86a8-4959d4984643 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:30.690837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151513d-3c97-4564-86a8-4959d4984643 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:30.695787 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:30.696404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:30.696681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:30.696947 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:30.697611 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:44:30.697908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:30.700250 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:30.713968 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1420/2845] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 14:44:30.720524 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aac7d-3e42-44b0-8415-9da1cdbacc87 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:30.733446 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aac7d-3e42-44b0-8415-9da1cdbacc87 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:30.738614 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:30.739246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:30.739520 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:30.739787 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:30.740444 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:30.742530 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:30.971397 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1426/2846] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:44:30 2022] POST /identity/v3/projects => generated 383 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:30.975918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:30.992832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beac06-4bac-4e82-b41a-c6f1af2ac595 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:31.062154 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:31.062784 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:31.063216 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:31.063568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:31.064196 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:31.066321 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:31.075693 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:31.203892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1421/2847] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:44:30 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:44:31.208683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:31.221672 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c32e9-d5d2-413b-bf87-e8ae31af5294 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:31.227033 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:31.227834 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:31.228214 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:31.228581 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:31.229440 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:31.232219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:31.258211 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1427/2848] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:44:31.263598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:31.284414 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf90d7-9e41-4afe-bb68-3665bff5df3d admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:31.293912 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae86b4-6849-419b-9b31-ea16644a5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:31.295078 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae86b4-6849-419b-9b31-ea16644a5e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:31.295596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae86b4-6849-419b-9b31-ea16644a5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:31.296104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae86b4-6849-419b-9b31-ea16644a5e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:31.297383 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae86b4-6849-419b-9b31-ea16644a5e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:31.300732 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae86b4-6849-419b-9b31-ea16644a5e4b admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:31.323865 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1422/2849] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:44:31.330460 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:31.351722 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdad84-48c8-4543-ac69-eda3cad8695a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:31.362078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:31.362078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:31.362078 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:31.362541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] PATH_INFO: `/v3/projects/094eba75e37c4f5dacca19f83df45066/users/b58ceb5012e547baa598b99880901f5e/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:31.387831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094eba75e37c4f5dacca19f83df45066, user_id=b58ceb5012e547baa598b99880901f5e, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:31.391957 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:31.489753 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1428/2850] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:44:31 2022] PUT /identity/v3/projects/094eba75e37c4f5dacca19f83df45066/users/b58ceb5012e547baa598b99880901f5e/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:44:31.495139 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f4c4-dfee-4041-8b41-57ae99207b71 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:31.495545 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f4c4-dfee-4041-8b41-57ae99207b71 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:31.495861 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f4c4-dfee-4041-8b41-57ae99207b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:31.523856 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-fe29f4c4-dfee-4041-8b41-57ae99207b71 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:31.525088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-fe29f4c4-dfee-4041-8b41-57ae99207b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:31.611102 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1423/2851] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:31.623119 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:31.624920 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:31.698762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e29d2-f36c-4007-8647-dd2017d057c4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:31.703382 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:31.704013 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:31.704643 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:31.706222 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:31.716697 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:31.718561 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:31.729463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1429/2852] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:33.334075 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e9c6b-4259-4d4b-963c-95ebb4e73b7e None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:33.335089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e9c6b-4259-4d4b-963c-95ebb4e73b7e None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:33.335089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e9c6b-4259-4d4b-963c-95ebb4e73b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:33.391233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-282e9c6b-4259-4d4b-963c-95ebb4e73b7e None None] MFA Rules not processed for user `b58ceb5012e547baa598b99880901f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:33.393552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-282e9c6b-4259-4d4b-963c-95ebb4e73b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:33.498302 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1424/2853] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:33.511023 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:33.513492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:33.533493 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:33.600151 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcefcb2-e3a3-432d-8f3c-4998df20129c service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:33.604269 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:33.604896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:33.605167 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:33.605488 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:33.615381 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:33.617008 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:33.627453 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1430/2854] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:44:35.712184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:35.712837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:35.712837 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:35.734211 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] MFA Rules not processed for user `b58ceb5012e547baa598b99880901f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:44:35.735424 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:35.743142 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1425/2855] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:44:40.917696 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:40.931682 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42d2cb-00c9-42d1-9011-1d4f54d8f8b4 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:40.938163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:40.939091 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:40.939436 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:40.939831 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:40.942105 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:41.023247 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:41.024685 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:41.037442 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1431/2856] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:44:44.906300 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:44:44.908266 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:44.924483 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0877b-915f-4671-9668-6204bb2ea9f4 None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:44:44.928175 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:44:44.928804 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:44:44.929087 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:44:44.929358 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:44:44.931220 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:44:44.947596 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:44:44.950188 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:44:44.968170 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1426/2857] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:44:51.031624 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=56 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:44:54.958231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:INSERT=3,keystone:SELECT=75 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:45:03.047805 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:45:03.057736 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf38631-6be9-408d-b5c9-bf920995dd39 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:45:03.062347 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:45:03.062980 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:45:03.063419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:45:03.063693 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:45:03.065714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:45:03.155485 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:45:03.156933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:45:03.168712 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1432/2858] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:45:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:45:13.163063 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=30 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:45:45.927518 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:45:45.929098 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:45:45.946505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe866a7-9fcc-4011-b65a-a472a3f26d0d service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:45:45.954295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e468a1-7c98-40e7-8576-c33949c6bc8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:45:45.954942 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e468a1-7c98-40e7-8576-c33949c6bc8d service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:45:45.955224 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e468a1-7c98-40e7-8576-c33949c6bc8d service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:45:45.955500 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e468a1-7c98-40e7-8576-c33949c6bc8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:45:45.958051 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b1e468a1-7c98-40e7-8576-c33949c6bc8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:45:45.975380 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e468a1-7c98-40e7-8576-c33949c6bc8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:45:45.977582 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e468a1-7c98-40e7-8576-c33949c6bc8d service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:45:45.993478 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1427/2859] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:45:50.434482 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:45:50.445429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86586f-7ceb-4bc3-b463-a7faf7d5001d service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:45:50.451766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:45:50.452601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:45:50.453210 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:45:50.453687 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:45:50.456357 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:45:50.468690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:45:50.470239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:45:50.481062 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1433/2860] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:45:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:45:55.987074 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:45:56.694245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d530ff-8e89-4ba6-a1d2-c3e66e75b3bf None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:45:56.694577 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d530ff-8e89-4ba6-a1d2-c3e66e75b3bf None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:45:56.694892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d530ff-8e89-4ba6-a1d2-c3e66e75b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:45:56.716400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-30d530ff-8e89-4ba6-a1d2-c3e66e75b3bf None None] MFA Rules not processed for user `b58ceb5012e547baa598b99880901f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:45:56.717644 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-30d530ff-8e89-4ba6-a1d2-c3e66e75b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:45:56.724836 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1428/2861] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:45:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:45:57.549335 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:45:57.551003 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:45:57.660739 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac180ad-3ce9-40e5-81da-ffc68ae8b783 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:45:57.666650 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:45:57.667384 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:45:57.667575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:45:57.667849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:45:57.669708 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:45:57.682747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:45:57.684370 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:45:57.696324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1434/2862] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:45:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:46:06.710594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:46:07.690853 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=38 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:46:08.782625 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d530ff-8e89-4ba6-a1d2-c3e66e75b3bf None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:46:08.799289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d530ff-8e89-4ba6-a1d2-c3e66e75b3bf None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:46:08.807979 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:46:08.808768 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:46:08.809104 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:46:08.809448 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:46:08.811632 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:46:08.826160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:46:08.827828 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:46:08.840668 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1429/2863] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:46:18.834782 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:47:08.343623 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:47:08.357840 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25edb7-6f8a-4f7d-8607-6d3f00692e66 service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:47:08.363714 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:47:08.364365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:47:08.364642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:47:08.364909 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:47:08.367674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:47:08.383879 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:47:08.385726 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:47:08.402927 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1435/2864] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:47:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:47:08.489612 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:47:08.509636 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c0659-2dab-4764-a8a0-3e6fd70cb542 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:47:08.515082 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:47:08.515793 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:47:08.516005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:47:08.516279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] PATH_INFO: `/v3/users/b58ceb5012e547baa598b99880901f5e` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:47:08.517539 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58ceb5012e547baa598b99880901f5e)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:47:08.519526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:47:08.807628 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58ceb5012e547baa598b99880901f5e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:08.808329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58ceb5012e547baa598b99880901f5e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:08.809088 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58ceb5012e547baa598b99880901f5e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:08.930875 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58ceb5012e547baa598b99880901f5e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:08.940550 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58ceb5012e547baa598b99880901f5e'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:08.967581 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1430/2865] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:47:08 2022] DELETE /identity/v3/users/b58ceb5012e547baa598b99880901f5e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:47:09.253144 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:47:09.268917 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154263cb-e0a9-49f0-b552-51f1869d6c59 service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:47:09.345094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:47:09.345858 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:47:09.346054 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:47:09.346328 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] PATH_INFO: `/v3/projects/094eba75e37c4f5dacca19f83df45066` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:47:09.347563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=094eba75e37c4f5dacca19f83df45066)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:47:09.349597 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:47:09.485239 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094eba75e37c4f5dacca19f83df45066', 'request_id': 'req-95b5535d-3389-4115-b321-c35b8ff51394'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:09.491890 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094eba75e37c4f5dacca19f83df45066', 'request_id': 'req-95b5535d-3389-4115-b321-c35b8ff51394'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:09.592201 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094eba75e37c4f5dacca19f83df45066', 'request_id': 'req-95b5535d-3389-4115-b321-c35b8ff51394'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:09.612674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] The token cache is being invalidate because project 094eba75e37c4f5dacca19f83df45066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:47:09.612674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:09.612674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:47:09.612674 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1436/2866] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:47:09 2022] DELETE /identity/v3/projects/094eba75e37c4f5dacca19f83df45066 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:47:18.961769 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:47:19.603505 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:47:28.962378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:47:28.964773 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:47:29.043485 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f274803-658a-47db-829f-867f797a2dce admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:47:29.120157 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:47:29.121249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:47:29.121748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:47:29.122289 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:47:29.124805 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:47:29.138117 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:47:29.139783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:47:29.149504 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1431/2867] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:47:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:47:29.247365 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:47:29.249163 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:47:29.264031 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5535d-3389-4115-b321-c35b8ff51394 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:47:29.267843 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:47:29.268469 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:47:29.268746 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:47:29.269021 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:47:29.270894 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:47:29.363463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:47:29.364960 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:47:29.380277 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1437/2868] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:47:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:47:39.145068 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=46 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:47:39.373748 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=34 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:47:54.316741 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:47:54.318907 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:47:54.410592 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bd78-40e9-4615-aa45-c0b5042a79de service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:47:54.416713 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:47:54.417458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:47:54.417785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:47:54.418133 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:47:54.420214 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:47:54.434073 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:47:54.435759 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:47:54.447895 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1432/2869] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:47:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:48:04.442333 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=32 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:05.543968 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:05.546723 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:05.561112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11228fc5-ef96-4b04-86d8-8b24c7c89a48 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:05.565219 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:05.565863 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:05.567577 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:05.567871 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:05.569937 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:05.668421 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:05.671070 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:05.686836 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1438/2870] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:49:15.295137 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:15.297392 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:15.321056 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baeb43-b2e9-4c8e-8d90-8ac27d55a10e service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:15.327874 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:15.328831 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:15.329263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:15.329752 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] PATH_INFO: `/v3/users/866e92f9dfb548e1830013af90281362` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:15.332260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=866e92f9dfb548e1830013af90281362)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:15.335010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:15.597537 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866e92f9dfb548e1830013af90281362'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:15.598552 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866e92f9dfb548e1830013af90281362'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:15.599259 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866e92f9dfb548e1830013af90281362'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:15.681290 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=30 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:15.686653 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866e92f9dfb548e1830013af90281362'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:15.697404 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866e92f9dfb548e1830013af90281362'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:15.721484 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1433/2871] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:49:15 2022] DELETE /identity/v3/users/866e92f9dfb548e1830013af90281362 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:49:16.008731 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:16.021369 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a4770-6f34-492a-be50-0929ebef7d29 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:16.089889 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:16.090569 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:16.090842 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:16.091107 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] PATH_INFO: `/v3/projects/d87a7f407bf64c4a9ed59037a569b0a1` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:16.092299 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87a7f407bf64c4a9ed59037a569b0a1)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:16.094225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:16.243331 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87a7f407bf64c4a9ed59037a569b0a1', 'request_id': 'req-0f5f064d-9588-4a80-908e-0502af399ab1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.252598 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87a7f407bf64c4a9ed59037a569b0a1', 'request_id': 'req-0f5f064d-9588-4a80-908e-0502af399ab1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.327110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87a7f407bf64c4a9ed59037a569b0a1', 'request_id': 'req-0f5f064d-9588-4a80-908e-0502af399ab1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.341891 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] The token cache is being invalidate because project d87a7f407bf64c4a9ed59037a569b0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:49:16.342256 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.342948 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.344854 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1439/2872] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:49:16 2022] DELETE /identity/v3/projects/d87a7f407bf64c4a9ed59037a569b0a1 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:49:16.349714 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:16.351288 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:16.366369 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709dcc2-50cf-477b-a2e3-19eeb9656284 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:16.430311 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:16.430965 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:16.431245 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:16.431523 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] PATH_INFO: `/v3/users/3c041e10d4a34cecb5638cca1898675d` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:16.433310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c041e10d4a34cecb5638cca1898675d)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:16.435318 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:16.640959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c041e10d4a34cecb5638cca1898675d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.641663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c041e10d4a34cecb5638cca1898675d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.642420 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c041e10d4a34cecb5638cca1898675d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.743774 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c041e10d4a34cecb5638cca1898675d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.752750 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-27ad7b10-1ebb-4cf2-965f-9054cc6e53fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c041e10d4a34cecb5638cca1898675d'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:16.762896 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:16.782263 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1434/2873] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:49:16 2022] DELETE /identity/v3/users/3c041e10d4a34cecb5638cca1898675d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:49:17.112242 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:17.114419 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:17.133245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f064d-9588-4a80-908e-0502af399ab1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:17.199491 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:17.200271 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:17.200627 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:17.201011 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] PATH_INFO: `/v3/projects/777cde9a912c4bc8999db494aae5bece` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:17.202473 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=777cde9a912c4bc8999db494aae5bece)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:17.204532 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:17.423213 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:17.426856 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777cde9a912c4bc8999db494aae5bece', 'request_id': 'req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:17.432961 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777cde9a912c4bc8999db494aae5bece', 'request_id': 'req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:17.630253 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777cde9a912c4bc8999db494aae5bece', 'request_id': 'req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:17.644403 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] The token cache is being invalidate because project 777cde9a912c4bc8999db494aae5bece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:49:17.644759 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:17.645464 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95e24acc-eadb-4d1c-b1d2-ec2d15de6863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:49:17.647124 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1440/2874] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:49:17 2022] DELETE /identity/v3/projects/777cde9a912c4bc8999db494aae5bece => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:49:17.655698 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9438245-2135-4c91-b1a5-9539ed38c92c None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:17.656306 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9438245-2135-4c91-b1a5-9539ed38c92c None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:17.656597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9438245-2135-4c91-b1a5-9539ed38c92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:17.693301 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-f9438245-2135-4c91-b1a5-9539ed38c92c None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:49:17.695477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f9438245-2135-4c91-b1a5-9539ed38c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:17.781891 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1435/2875] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:49:17.790762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a619be-3217-41a8-88a3-27b4f1d48cfd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:17.791186 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a619be-3217-41a8-88a3-27b4f1d48cfd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:17.791575 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a619be-3217-41a8-88a3-27b4f1d48cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:17.815175 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a4a619be-3217-41a8-88a3-27b4f1d48cfd None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:49:17.818850 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a4a619be-3217-41a8-88a3-27b4f1d48cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:17.828405 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1441/2876] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:49:17.834647 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9438245-2135-4c91-b1a5-9539ed38c92c None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:17.845443 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9438245-2135-4c91-b1a5-9539ed38c92c None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:17.851669 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:17.852295 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:17.852717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:17.852997 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:17.853675 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81725) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 14:49:17.853999 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:17.855706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:17.871275 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1436/2877] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 24 14:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 14:49:17.877639 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a619be-3217-41a8-88a3-27b4f1d48cfd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:17.889104 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a619be-3217-41a8-88a3-27b4f1d48cfd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:17.894523 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:17.895468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:17.895740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:17.896006 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:17.896661 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:17.898667 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:18.022326 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1442/2878] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:49:17 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:49:18.026883 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:18.038935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b021-e0d2-4610-b31b-be40f0597c9a admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:18.104884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:18.105505 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:18.105783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:18.106089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:18.106744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:18.108726 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:18.196727 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1437/2879] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:49:18 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:49:18.201181 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:18.213317 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892d1dc-cdbd-4931-af8c-682a3e5326f6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:18.218476 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:18.219110 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:18.219386 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:18.219655 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:18.220314 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:18.222747 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:18.235435 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1443/2880] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:49:18.240093 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:18.252024 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0344c12-bf82-4f9c-95e6-b23151648cdb admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:18.257540 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f894f-7fa0-46cf-a721-634d9fd584a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:18.258229 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f894f-7fa0-46cf-a721-634d9fd584a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:18.258508 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f894f-7fa0-46cf-a721-634d9fd584a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:18.258785 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f894f-7fa0-46cf-a721-634d9fd584a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:18.259457 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f894f-7fa0-46cf-a721-634d9fd584a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:18.261118 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f894f-7fa0-46cf-a721-634d9fd584a5 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:18.274844 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1438/2881] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:49:18.281492 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:18.294327 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aba7f-957a-472d-9e5f-7e2bde38893d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:18.299833 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:18.300906 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:18.301348 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:18.301632 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] PATH_INFO: `/v3/projects/1fbe0d75d24d4ce880f195bf1146d4c6/users/886d7594a0274190a409378517e326fb/roles/80a1f8f62542401a855036270bf2c51a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:18.321295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fbe0d75d24d4ce880f195bf1146d4c6, user_id=886d7594a0274190a409378517e326fb, role_id=80a1f8f62542401a855036270bf2c51a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:18.324467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:18.590664 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1444/2882] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:49:18 2022] PUT /identity/v3/projects/1fbe0d75d24d4ce880f195bf1146d4c6/users/886d7594a0274190a409378517e326fb/roles/80a1f8f62542401a855036270bf2c51a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:49:18.596325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edd8ea-32c1-4e4c-a83c-715539798ca8 None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:18.597160 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edd8ea-32c1-4e4c-a83c-715539798ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:18.597357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edd8ea-32c1-4e4c-a83c-715539798ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:18.619378 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-d2edd8ea-32c1-4e4c-a83c-715539798ca8 None None] MFA Rules not processed for user `34ed7567d900480cb00efd1e75c2ddc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:49:18.621242 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-d2edd8ea-32c1-4e4c-a83c-715539798ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:18.701600 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:18.713746 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1439/2883] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:49:18.724690 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:18.726789 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:18.796673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732384c-3a71-4867-bf74-f52100d85e6b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:18.802806 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:18.803857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:18.804279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:18.804780 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:18.820316 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:18.821782 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:18.837790 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1445/2884] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:49:25.449195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a42a57-4c69-43c9-b9da-5be87ed4311a None None] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:25.449681 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a42a57-4c69-43c9-b9da-5be87ed4311a None None] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:25.450184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a42a57-4c69-43c9-b9da-5be87ed4311a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:25.490935 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.auth.core [None req-c1a42a57-4c69-43c9-b9da-5be87ed4311a None None] MFA Rules not processed for user `886d7594a0274190a409378517e326fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81725) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:49:25.492125 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-c1a42a57-4c69-43c9-b9da-5be87ed4311a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:25.604568 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1440/2885] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:49:25.611153 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:25.625821 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923190-d3f8-4f95-aeaf-2c47b93a3166 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:25.631757 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:25.632340 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:25.632608 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:25.632886 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:25.633540 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:25.635809 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:25.733254 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1446/2886] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 24 14:49:25 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 14:49:25.739036 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a42a57-4c69-43c9-b9da-5be87ed4311a None None] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:25.752597 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a42a57-4c69-43c9-b9da-5be87ed4311a None None] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:25.838270 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:25.838975 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] REQUEST_METHOD: `POST` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:25.839257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:25.839531 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] PATH_INFO: `/v3/users` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:25.840240 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:25.842296 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:25.953694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1441/2887] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 24 14:49:25 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 14:49:25.959324 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:25.971669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60042daa-2a02-4ba5-8617-5ae207359421 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:25.977245 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:25.977897 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:25.978234 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:25.978463 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:25.979122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:25.980766 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:25.992572 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1447/2888] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 14:49:25.999717 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:26.019706 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c9e9f-36c9-4a8f-842d-3812f1854b04 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:26.027894 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:26.029256 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:26.029763 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:26.030325 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] PATH_INFO: `/v3/projects/4ef2d282df7a4e79a152e55bc0760b37/users/5ea656e44f2e46e8b2ad960da57714e8/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:26.050940 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef2d282df7a4e79a152e55bc0760b37, user_id=5ea656e44f2e46e8b2ad960da57714e8, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:26.053648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:26.137456 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1442/2889] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:49:25 2022] PUT /identity/v3/projects/4ef2d282df7a4e79a152e55bc0760b37/users/5ea656e44f2e46e8b2ad960da57714e8/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:49:26.142279 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:26.158162 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3710d9d-8dc2-4ee5-af8b-1b453a7626c4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:26.188289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:26.242803 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1984f-6a09-452a-824b-afb2dd96e5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:26.243566 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1984f-6a09-452a-824b-afb2dd96e5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:26.243880 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1984f-6a09-452a-824b-afb2dd96e5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:26.244194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1984f-6a09-452a-824b-afb2dd96e5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:26.245337 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1984f-6a09-452a-824b-afb2dd96e5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:26.247915 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1984f-6a09-452a-824b-afb2dd96e5c3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:26.264740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1448/2890] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 24 14:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 14:49:26.271460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:26.293158 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9453-3748-4000-a79c-27da49e751c1 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:26.298307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:26.299033 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:26.299310 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:26.299580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] PATH_INFO: `/v3/projects/4ef2d282df7a4e79a152e55bc0760b37/users/5ea656e44f2e46e8b2ad960da57714e8/roles/6dbae174ede84d6f9b9feaa4579bab05` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:26.302231 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef2d282df7a4e79a152e55bc0760b37, user_id=5ea656e44f2e46e8b2ad960da57714e8, role_id=6dbae174ede84d6f9b9feaa4579bab05)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:26.304918 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:26.317200 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1443/2891] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 24 14:49:26 2022] PUT /identity/v3/projects/4ef2d282df7a4e79a152e55bc0760b37/users/5ea656e44f2e46e8b2ad960da57714e8/roles/6dbae174ede84d6f9b9feaa4579bab05 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:49:33.963765 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e826d2b-672a-47cb-ad68-9fc12f17c1ef None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:33.964654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e826d2b-672a-47cb-ad68-9fc12f17c1ef None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:33.964654 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e826d2b-672a-47cb-ad68-9fc12f17c1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:34.014188 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3e826d2b-672a-47cb-ad68-9fc12f17c1ef None None] MFA Rules not processed for user `5ea656e44f2e46e8b2ad960da57714e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:49:34.015452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3e826d2b-672a-47cb-ad68-9fc12f17c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:34.118533 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1449/2892] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 14:49:34.135992 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:34.138185 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:34.194089 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:34.229418 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2f570-91ec-4db9-8b4b-16230d44a011 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:34.236161 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:34.236832 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:34.237648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:34.238748 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:34.303403 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:34.305047 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:34.317908 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1444/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:49:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:49:44.114265 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=51 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:44.491783 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=38 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:49:54.208562 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077a2ac-e50b-4e14-9d4c-d62bc4f5cfc3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:54.209197 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077a2ac-e50b-4e14-9d4c-d62bc4f5cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:54.209380 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077a2ac-e50b-4e14-9d4c-d62bc4f5cfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:54.239378 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3077a2ac-e50b-4e14-9d4c-d62bc4f5cfc3 None None] MFA Rules not processed for user `886d7594a0274190a409378517e326fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:49:54.242525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3077a2ac-e50b-4e14-9d4c-d62bc4f5cfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:54.256246 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1450/2894] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:49:54.322460 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:54.324071 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:54.401549 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623380a3-2f5b-4287-b6f0-4a421fb46064 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:54.405666 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9daf5be8a3dc48e78294882c62b8e60f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:54.406342 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:54.406619 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:54.406892 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:54.409387 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:54.425171 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:54.426961 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:54.440590 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1445/2895] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:49:54.795495 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077a2ac-e50b-4e14-9d4c-d62bc4f5cfc3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:54.880534 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077a2ac-e50b-4e14-9d4c-d62bc4f5cfc3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:54.884548 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:54.885187 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:54.885468 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:54.885743 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:54.887786 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:54.901195 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:54.903837 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:54.916484 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1451/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:49:56.942267 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:49:56.944148 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:56.958347 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7fae3-0f93-4dba-b910-b4dcb65ada82 service cinder] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:49:56.962323 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:49:56.962945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:49:56.963222 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:49:56.963493 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:49:56.966195 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:49:56.989594 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:49:56.991697 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:49:57.005873 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1446/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:49:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:50:04.911884 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=29 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:50:06.999925 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=32 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:50:16.798792 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:50:16.809024 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b31f-5a85-4a6e-abc7-8a5cbc642f2d service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:50:16.813404 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:50:16.818283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:50:16.818283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:50:16.818283 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:50:16.827258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:50:16.828932 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:50:16.841535 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1452/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:50:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:50:19.372578 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:50:19.378099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:50:19.393023 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c142deb-d75f-47c3-b9e0-193e20ab75c1 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:50:19.399121 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:50:19.399749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:50:19.400027 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:50:19.400294 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:50:19.402202 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:50:19.417112 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:50:19.418881 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:50:19.431635 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1447/2899] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:50:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:50:26.836115 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:50:29.426466 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:50:53.081122 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:50:53.090744 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c91e9f-0cbf-465d-9862-39c733241492 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:50:53.095423 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf021b-d000-4f9b-9dc3-6156b99e8e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:50:53.096056 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf021b-d000-4f9b-9dc3-6156b99e8e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:50:53.096336 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf021b-d000-4f9b-9dc3-6156b99e8e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:50:53.096610 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf021b-d000-4f9b-9dc3-6156b99e8e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:50:53.098541 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f6cf021b-d000-4f9b-9dc3-6156b99e8e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:50:53.110525 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf021b-d000-4f9b-9dc3-6156b99e8e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:50:53.112418 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf021b-d000-4f9b-9dc3-6156b99e8e84 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:50:53.122509 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1453/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:50:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:51:03.117711 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:52:17.848124 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:52:17.927408 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30ff68-08b2-4f86-8eb8-ddbaacb15e5a service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:52:17.936064 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:52:17.940794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:52:17.940794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:52:17.940794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:52:17.940794 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:52:17.984108 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:52:17.985694 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:52:18.000744 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1448/2901] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:52:19.960719 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a8df-96ea-4290-8928-cb67fb8bb368 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:52:19.961294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a8df-96ea-4290-8928-cb67fb8bb368 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:52:19.961294 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a8df-96ea-4290-8928-cb67fb8bb368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:52:19.983601 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dd24a8df-96ea-4290-8928-cb67fb8bb368 None None] MFA Rules not processed for user `886d7594a0274190a409378517e326fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:52:19.985258 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dd24a8df-96ea-4290-8928-cb67fb8bb368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:52:19.992777 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1454/2902] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:52:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:52:21.258598 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:52:21.260081 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:52:21.348489 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e35de-1d83-4fdd-ae42-3be07bac0f33 service nova] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:52:21.355595 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:52:21.357113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:52:21.357588 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:52:21.357963 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:52:21.360547 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:52:21.376618 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:52:21.378281 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:52:21.389329 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1449/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:52:29.977452 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:52:31.384580 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=60 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:52:36.752429 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a8df-96ea-4290-8928-cb67fb8bb368 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:52:36.772406 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a8df-96ea-4290-8928-cb67fb8bb368 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:52:36.779111 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fe1cc-3ece-4a5a-8a9f-46c8cd28ffcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '613e985e94c54913a5e6b93367157fc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:52:36.779742 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fe1cc-3ece-4a5a-8a9f-46c8cd28ffcd service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:52:36.780017 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fe1cc-3ece-4a5a-8a9f-46c8cd28ffcd service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:52:36.780288 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fe1cc-3ece-4a5a-8a9f-46c8cd28ffcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:52:36.782379 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d81fe1cc-3ece-4a5a-8a9f-46c8cd28ffcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:52:36.796295 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fe1cc-3ece-4a5a-8a9f-46c8cd28ffcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:52:36.797896 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fe1cc-3ece-4a5a-8a9f-46c8cd28ffcd service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:52:36.810683 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1455/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:52:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:52:46.805036 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:52:59.355054 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:52:59.384983 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a45b3-09c1-4160-8612-f7efd0b4b4ae service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:52:59.391770 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:52:59.392415 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:52:59.392695 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:52:59.392972 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:52:59.394930 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:52:59.429735 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:52:59.431438 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:52:59.445375 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1450/2905] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:52:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:53:09.439019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=15 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:54:18.313919 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6561375-ce03-4e72-b986-3d63233e8987 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:54:18.314640 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6561375-ce03-4e72-b986-3d63233e8987 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:54:18.314893 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6561375-ce03-4e72-b986-3d63233e8987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:54:18.342323 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c6561375-ce03-4e72-b986-3d63233e8987 None None] MFA Rules not processed for user `886d7594a0274190a409378517e326fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 14:54:18.343756 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c6561375-ce03-4e72-b986-3d63233e8987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:54:18.351621 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1456/2906] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 24 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 14:54:18.731471 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:54:18.755184 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78380c7-69bf-4182-ab6d-3587653a3ab2 service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:54:18.759252 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:54:18.760028 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:54:18.760377 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:54:18.760656 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:54:18.762661 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:54:18.788307 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:54:18.789803 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:54:18.799910 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1451/2907] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:54:28.335507 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:54:28.795724 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=15 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:54:33.829586 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6561375-ce03-4e72-b986-3d63233e8987 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:54:33.842849 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6561375-ce03-4e72-b986-3d63233e8987 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:54:33.846835 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:54:33.847522 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:54:33.847797 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:54:33.848094 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:54:33.858956 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:54:33.860441 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:54:33.873644 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1457/2908] 10.222.0.49 () {60 vars in 1396 bytes} [Wed Aug 24 14:54:33 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:54:43.866807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:55:56.084573 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:55:56.093845 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82012c8-8c6c-4502-8ae8-3e9435f8c85f service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:55:56.097869 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:55:56.098526 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:55:56.098802 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:55:56.099070 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:55:56.109290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:55:56.110945 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:55:56.121010 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1452/2909] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:55:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:56:06.116663 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:56:10.555009 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:10.565043 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3fd37-500c-4d38-ad29-10060ab404b3 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:10.569941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:10.572480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:10.572480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:10.572480 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:10.601807 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:10.603321 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:10.630225 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1458/2910] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:56:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:56:11.848400 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:11.858959 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3344a-1028-423a-a775-2c033e42e0a2 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:11.863233 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:11.863851 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:11.864130 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:11.865053 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:11.877003 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:11.878712 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:11.891458 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1453/2911] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:56:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:56:20.627112 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=13 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:56:21.885877 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=6 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:56:24.964635 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:24.975537 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b0c9-658f-421f-a185-c0e79c0904f9 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:24.979467 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48364ccf19434c4c9c09b4e5ce4bfc7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:24.980087 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:24.980360 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:24.980630 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:25.067642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:25.069229 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:25.092133 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1459/2912] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 14:56:29.594757 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:29.607379 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da669-82e0-4d46-ade9-eb4580dfe412 service neutron] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:29.613029 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:29.613739 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] REQUEST_METHOD: `GET` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:29.614585 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:29.615005 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:29.617113 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:56:29.630455 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:29.632174 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:29.644781 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1454/2913] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:56:34.081074 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:34.086130 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b9ec5-475f-4d22-9f6e-dd14fb69469d service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:34.091673 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:34.092359 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:34.092659 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:34.092933 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] PATH_INFO: `/v3/users/886d7594a0274190a409378517e326fb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:34.094095 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=886d7594a0274190a409378517e326fb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:34.096072 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:34.418857 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886d7594a0274190a409378517e326fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:34.419989 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886d7594a0274190a409378517e326fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:34.434416 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886d7594a0274190a409378517e326fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:34.435470 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886d7594a0274190a409378517e326fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:34.592718 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886d7594a0274190a409378517e326fb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:34.612729 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1460/2914] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:56:34 2022] DELETE /identity/v3/users/886d7594a0274190a409378517e326fb => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:56:34.914889 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:34.927357 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4ad64-a1ac-407d-af37-6bec7add64d6 service placement] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:34.991464 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:34.991978 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:34.992260 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:34.992608 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] PATH_INFO: `/v3/projects/1fbe0d75d24d4ce880f195bf1146d4c6` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:34.993683 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fbe0d75d24d4ce880f195bf1146d4c6)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:34.995648 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:35.137604 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fbe0d75d24d4ce880f195bf1146d4c6', 'request_id': 'req-742bccd9-8955-473c-a914-77686e182ad4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.143249 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fbe0d75d24d4ce880f195bf1146d4c6', 'request_id': 'req-742bccd9-8955-473c-a914-77686e182ad4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.149799 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fbe0d75d24d4ce880f195bf1146d4c6', 'request_id': 'req-742bccd9-8955-473c-a914-77686e182ad4'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.280811 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] The token cache is being invalidate because project 1fbe0d75d24d4ce880f195bf1146d4c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:56:35.281100 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.281747 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.283290 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1455/2915] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:56:34 2022] DELETE /identity/v3/projects/1fbe0d75d24d4ce880f195bf1146d4c6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 14:56:35.287841 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:35.289611 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:56:35.304018 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b1f01-0496-4152-ade7-0d3803b57936 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:35.354923 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:56:35.367642 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:35.368304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:35.368579 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:35.368845 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] PATH_INFO: `/v3/users/5ea656e44f2e46e8b2ad960da57714e8` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:35.370194 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea656e44f2e46e8b2ad960da57714e8)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:35.372000 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:35.608426 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea656e44f2e46e8b2ad960da57714e8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.609501 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea656e44f2e46e8b2ad960da57714e8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.617267 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea656e44f2e46e8b2ad960da57714e8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.617954 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea656e44f2e46e8b2ad960da57714e8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.768563 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea656e44f2e46e8b2ad960da57714e8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:35.788352 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1461/2916] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 24 14:56:35 2022] DELETE /identity/v3/users/5ea656e44f2e46e8b2ad960da57714e8 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:56:36.067413 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Authenticating user token {{(pid=81725) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:56:36.069627 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.fernet_utils [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81725) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:56:36.088038 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bccd9-8955-473c-a914-77686e182ad4 admin admin] Validating token access rules against request {{(pid=81725) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:56:36.149099 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34ed7567d900480cb00efd1e75c2ddc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f90d0e4c756a418fb562e2e8abdd6ef4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81725) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:56:36.149664 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:56:36.149952 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:56:36.150257 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] PATH_INFO: `/v3/projects/4ef2d282df7a4e79a152e55bc0760b37` {{(pid=81725) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:56:36.151429 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef2d282df7a4e79a152e55bc0760b37)` {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:56:36.153366 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] RBAC: Authorization granted {{(pid=81725) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:56:36.164043 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:56:36.376884 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef2d282df7a4e79a152e55bc0760b37', 'request_id': 'req-5843b897-5de2-468c-851e-82f400b7f347'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:36.383749 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef2d282df7a4e79a152e55bc0760b37', 'request_id': 'req-5843b897-5de2-468c-851e-82f400b7f347'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:36.390517 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef2d282df7a4e79a152e55bc0760b37', 'request_id': 'req-5843b897-5de2-468c-851e-82f400b7f347'} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:36.472696 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] The token cache is being invalidate because project 4ef2d282df7a4e79a152e55bc0760b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81725) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 14:56:36.473477 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:36.474279 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG keystone.notifications [None req-5843b897-5de2-468c-851e-82f400b7f347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81725) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 14:56:36.475809 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: [pid: 81725|app: 0|req: 1456/2917] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 24 14:56:36 2022] DELETE /identity/v3/projects/4ef2d282df7a4e79a152e55bc0760b37 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 14:56:46.517282 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:56:46.810019 dv-cinder-756709-6-iscsi devstack@keystone.service[81725]: DEBUG dbcounter [-] [81725] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 14:58:01.436785 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 14:58:01.438304 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:58:01.522472 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae167c-7a49-4fb6-b80b-d9d29d13d6ac admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 14:58:01.584762 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66135a-8c45-48cb-bba3-57937cc254a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739aa6ea7c043d999a3280deb9963d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4f2fa9f48864e54b67f4c732710b225', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 14:58:01.585388 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66135a-8c45-48cb-bba3-57937cc254a2 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 14:58:01.585669 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66135a-8c45-48cb-bba3-57937cc254a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 14:58:01.585941 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66135a-8c45-48cb-bba3-57937cc254a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 14:58:01.587752 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5a66135a-8c45-48cb-bba3-57937cc254a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 14:58:01.600289 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66135a-8c45-48cb-bba3-57937cc254a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 14:58:01.601868 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66135a-8c45-48cb-bba3-57937cc254a2 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 14:58:01.613342 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1462/2918] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 24 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 14:58:11.607740 dv-cinder-756709-6-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=58 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}